2024-09-19T21:21:56.958 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T21:21:56.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T21:21:57.053 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850 branch: reef description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909850' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 4623 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 80/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_22:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T21:21:57.053 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa; will attempt to use it 2024-09-19T21:21:57.054 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa/tasks 2024-09-19T21:21:57.054 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T21:21:57.056 INFO:teuthology.task.internal:Checking packages... 2024-09-19T21:21:57.076 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-19T21:21:57.077 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T21:21:57.077 INFO:teuthology.packaging:ref: None 2024-09-19T21:21:57.077 INFO:teuthology.packaging:tag: None 2024-09-19T21:21:57.077 INFO:teuthology.packaging:branch: reef 2024-09-19T21:21:57.077 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:21:57.077 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-09-19T21:21:57.307 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-867-g01eb14ce-1focal 2024-09-19T21:21:57.308 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T21:21:57.315 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T21:21:57.315 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T21:21:57.322 INFO:teuthology.task.internal:Saving configuration 2024-09-19T21:21:57.334 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T21:21:57.341 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T21:21:57.363 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 21:15:44.243941', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-19T21:21:57.381 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 21:15:44.244724', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-19T21:21:57.401 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 21:15:44.242625', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-19T21:21:57.402 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T21:21:57.408 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-19T21:21:57.408 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-19T21:21:57.408 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['client.0'] 2024-09-19T21:21:57.408 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T21:21:57.483 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd8d7247010>, signals=[15]) 2024-09-19T21:21:57.484 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T21:21:57.491 INFO:teuthology.task.internal:Opening connections... 2024-09-19T21:21:57.491 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:21:57.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:21:57.592 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:21:57.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:21:57.681 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-19T21:21:57.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:21:57.769 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T21:21:57.777 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-09-19T21:21:57.784 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-09-19T21:21:57.784 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-09-19T21:21:57.830 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2024-09-19T21:21:57.830 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="20.04" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=focal 2024-09-19T21:21:57.831 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=focal 2024-09-19T21:21:57.832 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-09-19T21:21:57.855 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-09-19T21:21:57.862 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-09-19T21:21:57.862 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Ubuntu" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:ID=ubuntu 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE=debian 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="20.04" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T21:21:57.916 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_CODENAME=focal 2024-09-19T21:21:57.917 INFO:teuthology.orchestra.run.smithi033.stdout:UBUNTU_CODENAME=focal 2024-09-19T21:21:57.917 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-09-19T21:21:57.931 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-19T21:21:57.937 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-19T21:21:57.937 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="20.04" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T21:21:57.984 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=focal 2024-09-19T21:21:57.985 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=focal 2024-09-19T21:21:57.985 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-19T21:21:58.006 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T21:21:58.014 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T21:21:58.018 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T21:21:58.019 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-09-19T21:21:58.021 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-09-19T21:21:58.023 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-19T21:21:58.028 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T21:21:58.034 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T21:21:58.034 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-09-19T21:21:58.036 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-09-19T21:21:58.038 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-19T21:21:58.052 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T21:21:58.115 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T21:21:58.129 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T21:21:58.129 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T21:21:58.129 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T21:21:58.130 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T21:21:58.130 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T21:21:58.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:21:58.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:21:58.136 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-19T21:21:58.137 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-19T21:21:58.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:21:58.138 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-19T21:21:58.143 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-124-generic 2024-09-19T21:21:58.143 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.4.0-124-generic 2024-09-19T21:21:58.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-19T21:21:58.144 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-124-generic 2024-09-19T21:21:58.145 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.4.0-124-generic 2024-09-19T21:21:58.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-09-19T21:21:58.146 INFO:teuthology.orchestra.run.smithi122.stdout:5.4.0-124-generic 2024-09-19T21:21:58.146 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.4.0-124-generic 2024-09-19T21:21:58.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-19T21:21:58.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-19T21:21:58.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-19T21:21:58.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-19T21:21:58.422 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:21:58.443 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-19T21:21:58.447 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:21:58.449 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-19T21:21:58.449 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-19T21:21:58.465 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-19T21:21:58.500 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-19T21:21:58.512 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-19T21:21:58.537 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-19T21:21:58.573 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:21:58.655 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-19T21:21:58.750 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [818 kB] 2024-09-19T21:21:58.814 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,038 kB] 2024-09-19T21:21:58.838 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,227 kB] 2024-09-19T21:21:58.841 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,227 kB] 2024-09-19T21:21:58.886 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [476 kB] 2024-09-19T21:21:58.886 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-19T21:21:58.886 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,156 kB] 2024-09-19T21:21:58.893 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,601 kB] 2024-09-19T21:21:58.906 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-19T21:21:58.907 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-19T21:21:58.912 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:58.913 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-19T21:21:58.916 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-19T21:21:58.921 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-19T21:21:58.922 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-19T21:21:58.923 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-19T21:21:58.923 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-19T21:21:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [554 kB] 2024-09-19T21:21:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-19T21:21:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-19T21:21:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,283 kB] 2024-09-19T21:21:58.944 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-19T21:21:58.944 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-19T21:21:58.965 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [460 kB] 2024-09-19T21:21:58.969 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:58.969 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,232 kB] 2024-09-19T21:21:58.978 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-19T21:21:58.984 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-19T21:21:58.987 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-19T21:21:58.988 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-19T21:21:58.988 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-19T21:21:58.988 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-19T21:21:58.991 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-19T21:21:58.991 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-19T21:21:58.991 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-19T21:21:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,038 kB] 2024-09-19T21:21:59.002 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [818 kB] 2024-09-19T21:21:59.007 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-19T21:21:59.008 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-19T21:21:59.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-19T21:21:59.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-19T21:21:59.009 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-19T21:21:59.017 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [476 kB] 2024-09-19T21:21:59.021 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-19T21:21:59.022 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,156 kB] 2024-09-19T21:21:59.044 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-19T21:21:59.045 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-19T21:21:59.048 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:59.048 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-19T21:21:59.054 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-19T21:21:59.061 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-19T21:21:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-19T21:21:59.063 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-19T21:21:59.063 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-19T21:21:59.069 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-19T21:21:59.070 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-19T21:21:59.070 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,601 kB] 2024-09-19T21:21:59.109 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [554 kB] 2024-09-19T21:21:59.113 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-19T21:21:59.113 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-19T21:21:59.113 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,283 kB] 2024-09-19T21:21:59.135 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [460 kB] 2024-09-19T21:21:59.138 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:59.138 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,232 kB] 2024-09-19T21:21:59.141 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [818 kB] 2024-09-19T21:21:59.147 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-19T21:21:59.147 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,038 kB] 2024-09-19T21:21:59.152 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-19T21:21:59.154 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-19T21:21:59.154 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-19T21:21:59.155 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-19T21:21:59.155 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-19T21:21:59.160 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-19T21:21:59.160 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-19T21:21:59.161 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-19T21:21:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-19T21:21:59.177 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-19T21:21:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-19T21:21:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-19T21:21:59.393 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,601 kB] 2024-09-19T21:21:59.477 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,227 kB] 2024-09-19T21:21:59.560 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [554 kB] 2024-09-19T21:21:59.572 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-19T21:21:59.573 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-19T21:21:59.574 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,283 kB] 2024-09-19T21:21:59.667 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [460 kB] 2024-09-19T21:21:59.668 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [476 kB] 2024-09-19T21:21:59.682 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-19T21:21:59.683 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,156 kB] 2024-09-19T21:21:59.688 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:59.689 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,232 kB] 2024-09-19T21:21:59.718 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-19T21:21:59.740 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-19T21:21:59.747 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-19T21:21:59.748 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-19T21:21:59.749 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-19T21:21:59.749 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-19T21:21:59.777 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-19T21:21:59.778 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-19T21:21:59.792 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-19T21:21:59.792 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-19T21:21:59.810 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-19T21:21:59.812 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-19T21:21:59.813 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-19T21:21:59.814 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-19T21:21:59.815 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-19T21:21:59.841 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-19T21:21:59.848 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-19T21:21:59.849 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-19T21:21:59.849 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-19T21:21:59.894 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-19T21:21:59.895 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-19T21:21:59.895 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-19T21:21:59.914 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-19T21:21:59.914 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-19T21:22:03.380 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 22.0 MB in 3s (7,308 kB/s) 2024-09-19T21:22:03.410 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 22.0 MB in 3s (7,229 kB/s) 2024-09-19T21:22:03.863 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 22.0 MB in 3s (6,809 kB/s) 2024-09-19T21:22:04.948 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:22:04.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:22:04.980 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:22:05.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:22:05.074 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:22:05.081 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:22:05.299 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-19T21:22:05.299 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-19T21:22:05.301 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-19T21:22:05.302 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-19T21:22:05.480 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:22:05.502 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-19T21:22:05.503 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:05.503 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-19T21:22:05.503 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:05.504 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-19T21:22:05.504 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-19T21:22:05.504 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-19T21:22:05.504 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-09-19T21:22:05.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:22:05.510 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-19T21:22:05.510 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:05.510 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-19T21:22:05.510 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:05.511 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-19T21:22:05.511 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-19T21:22:05.511 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-19T21:22:05.511 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2024-09-19T21:22:05.559 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-19T21:22:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:05.559 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-19T21:22:05.560 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:05.560 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-19T21:22:05.561 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T21:22:05.574 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-19T21:22:05.574 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:05.574 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-19T21:22:05.574 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:05.576 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-09-19T21:22:05.577 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T21:22:05.605 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:22:05.615 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:22:05.615 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 77.2 MB of archives. 2024-09-19T21:22:05.615 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-19T21:22:05.615 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-19T21:22:05.632 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:22:05.632 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 77.2 MB of archives. 2024-09-19T21:22:05.632 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-19T21:22:05.632 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-19T21:22:05.835 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-19T21:22:05.835 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-19T21:22:05.933 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-19T21:22:05.953 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-19T21:22:06.025 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-19T21:22:06.047 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-19T21:22:06.065 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-09-19T21:22:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-19T21:22:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:06.067 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-09-19T21:22:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-19T21:22:06.067 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-09-19T21:22:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: thermald 2024-09-19T21:22:06.129 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-09-19T21:22:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-19T21:22:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-19T21:22:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-19T21:22:06.131 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-09-19T21:22:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T21:22:06.313 INFO:teuthology.orchestra.run.smithi033.stdout:3 upgraded, 5 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:22:06.314 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 77.2 MB of archives. 2024-09-19T21:22:06.314 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-19T21:22:06.314 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-19T21:22:06.531 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-19T21:22:06.531 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-19T21:22:06.532 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-19T21:22:06.543 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-19T21:22:06.543 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-19T21:22:06.544 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-19T21:22:06.640 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-19T21:22:06.651 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-19T21:22:06.731 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-19T21:22:06.750 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-19T21:22:07.332 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 77.2 MB in 1s (72.3 MB/s) 2024-09-19T21:22:07.492 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-19T21:22:07.519 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 77.2 MB in 1s (65.3 MB/s) 2024-09-19T21:22:07.525 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-19T21:22:07.709 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-19T21:22:07.767 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-19T21:22:09.122 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-19T21:22:09.123 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-19T21:22:09.123 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-19T21:22:09.276 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-19T21:22:09.281 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:09.403 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-19T21:22:09.428 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:09.447 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-19T21:22:09.452 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:09.454 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-19T21:22:09.581 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:10.197 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 77.2 MB in 3s (23.3 MB/s) 2024-09-19T21:22:10.379 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-19T21:22:11.390 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-19T21:22:11.405 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:11.498 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:11.573 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-19T21:22:11.586 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:11.699 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:12.087 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-19T21:22:12.092 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:12.360 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:12.626 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-19T21:22:12.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:12.676 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:12.801 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-19T21:22:12.815 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:12.859 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:14.502 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-19T21:22:14.516 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:14.612 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:15.747 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-19T21:22:15.760 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:15.797 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:17.848 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:17.880 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:18.000 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:18.336 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:18.375 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:18.461 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:18.501 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:18.815 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-19T21:22:18.830 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-19T21:22:18.854 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-19T21:22:18.868 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-19T21:22:18.873 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:18.912 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:20.887 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:21.022 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:21.372 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:21.500 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:21.854 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-19T21:22:21.868 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-19T21:22:21.912 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:24.535 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-19T21:22:24.556 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:24.593 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:25.150 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-19T21:22:25.171 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:25.216 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:27.589 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-19T21:22:27.603 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:27.609 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-19T21:22:27.680 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:27.725 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:27.964 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:28.076 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:28.118 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:28.205 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:28.364 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-19T21:22:28.444 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:28.507 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:28.607 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:28.733 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-19T21:22:28.834 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:29.825 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:30.688 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:31.012 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-19T21:22:31.140 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-19T21:22:31.453 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:22:31.453 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:31.492 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-19T21:22:31.620 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:31.770 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-19T21:22:31.851 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:22:31.851 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:31.897 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:32.000 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:32.228 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:33.173 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:33.210 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-19T21:22:33.336 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-19T21:22:33.437 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:33.445 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-19T21:22:33.529 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T21:22:33.529 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:33.572 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-19T21:22:33.740 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:33.823 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T21:22:33.823 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:34.342 INFO:teuthology.orchestra.run.smithi033.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:22:34.342 INFO:teuthology.orchestra.run.smithi033.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:35.776 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:37.536 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-19T21:22:37.687 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-19T21:22:37.813 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-19T21:22:37.887 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T21:22:37.887 INFO:teuthology.orchestra.run.smithi033.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:55.916 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T21:22:55.916 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-09-19T21:22:55.944 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:22:55.964 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-09-19T21:22:56.271 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T21:22:56.271 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2024-09-19T21:22:56.281 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:22:56.291 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2024-09-19T21:22:56.300 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:22:56.319 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:56.594 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:22:56.623 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:22:56.737 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:22:56.748 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:22:57.034 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:22:57.046 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:22:59.526 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-09-19T21:22:59.791 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-09-19T21:22:59.967 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-19T21:22:59.996 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T21:22:59.996 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub' 2024-09-19T21:23:00.008 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:23:00.024 INFO:teuthology.orchestra.run.smithi033.stdout:Generating grub configuration file ... 2024-09-19T21:23:00.028 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 20 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:00.029 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-19T21:23:00.030 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:00.030 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:00.030 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-19T21:23:00.030 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-19T21:23:00.030 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T21:23:00.030 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T21:23:00.030 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T21:23:00.031 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T21:23:00.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-19T21:23:00.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-19T21:23:00.237 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:23:00.253 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-19T21:23:00.310 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-19T21:23:00.310 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:00.311 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-19T21:23:00.312 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-19T21:23:00.312 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T21:23:00.312 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T21:23:00.312 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T21:23:00.312 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T21:23:00.312 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-19T21:23:00.378 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:23:00.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-19T21:23:00.456 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:23:00.520 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:23:00.520 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:23:00.521 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:23:00.525 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:23:00.538 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:23:00.542 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:23:00.549 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:23:00.930 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:23:00.941 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:23:02.068 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:23:02.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:23:02.168 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:23:02.387 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-19T21:23:02.387 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-19T21:23:02.441 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:23:02.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:23:02.541 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:23:02.628 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-19T21:23:02.628 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:23:02.630 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 20 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-19T21:23:02.651 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:02.652 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-19T21:23:02.652 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-19T21:23:02.661 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.JJfKJR30bS 2024-09-19T21:23:02.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.JJfKJR30bS 2024-09-19T21:23:02.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.JJfKJR30bS 2024-09-19T21:23:02.721 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.JJfKJR30bS is 9KB 2024-09-19T21:23:02.736 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.JJfKJR30bS 2024-09-19T21:23:02.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:02.770 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-19T21:23:02.770 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-19T21:23:02.799 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:23:02.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T21:23:02.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:02.827 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-19T21:23:02.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-09-19T21:23:03.020 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-19T21:23:03.020 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:23:03.022 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-19T21:23:03.043 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-19T21:23:03.043 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-19T21:23:03.043 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-19T21:23:03.044 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:03.045 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:03.045 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-19T21:23:03.045 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-19T21:23:03.050 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.Wh1ToSUP9k 2024-09-19T21:23:03.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Wh1ToSUP9k 2024-09-19T21:23:03.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.Wh1ToSUP9k 2024-09-19T21:23:03.107 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.Wh1ToSUP9k is 9KB 2024-09-19T21:23:03.120 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.Wh1ToSUP9k 2024-09-19T21:23:03.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:03.185 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:23:03.185 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T21:23:03.185 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:03.215 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-19T21:23:03.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2024-09-19T21:23:03.834 INFO:teuthology.orchestra.run.smithi033.stdout:done 2024-09-19T21:23:04.263 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-09-19T21:23:04.303 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-19T21:23:04.308 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:23:04.322 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:04.327 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-09-19T21:23:04.328 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:04.328 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:04.328 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-09-19T21:23:04.328 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-19T21:23:04.328 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T21:23:04.328 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T21:23:04.328 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T21:23:04.328 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T21:23:04.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-09-19T21:23:04.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-19T21:23:04.553 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:23:04.576 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:23:04.588 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:23:04.669 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:23:04.722 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2024-09-19T21:23:04.729 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:23:04.741 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-09-19T21:23:04.750 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:23:04.832 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:23:04.983 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:23:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:23:04.994 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:23:05.004 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:23:05.393 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:23:05.405 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:23:06.487 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:23:06.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T21:23:06.625 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:23:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-19T21:23:06.668 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-19T21:23:06.849 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-19T21:23:06.849 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-19T21:23:07.091 INFO:teuthology.orchestra.run.smithi033.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-19T21:23:07.091 INFO:teuthology.orchestra.run.smithi033.stdout:0 upgraded, 0 newly installed, 0 to remove and 318 not upgraded. 2024-09-19T21:23:07.092 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-09-19T21:23:07.113 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-09-19T21:23:07.113 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-09-19T21:23:07.113 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 20 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.4.0.196.194 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-09-19T21:23:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-09-19T21:23:07.115 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-19T21:23:07.121 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.PIpWeNa6d3 2024-09-19T21:23:07.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PIpWeNa6d3 2024-09-19T21:23:07.130 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-09-19T21:23:07.132 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-09-19T21:23:07.133 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.PIpWeNa6d3 2024-09-19T21:23:07.221 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.PIpWeNa6d3 is 9KB 2024-09-19T21:23:07.235 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.PIpWeNa6d3 2024-09-19T21:23:07.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:07.254 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:23:07.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T21:23:07.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T21:23:07.295 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-19T21:23:07.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo update-grub 2024-09-19T21:23:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub' 2024-09-19T21:23:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T21:23:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-09-19T21:23:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-19T21:23:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-19T21:23:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-19T21:23:09.506 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-19T21:23:11.178 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-09-19T21:23:11.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-09-19T21:23:36.693 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T21:23:36.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:23:36.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:23:37.136 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T21:23:37.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:23:37.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:23:41.185 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T21:23:41.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:23:41.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:23:55.069 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-19T21:23:55.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-19T21:23:59.677 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-19T21:24:04.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:24:04.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:04.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:24:04.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:07.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-19T21:24:08.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:24:08.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:19.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:24:19.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:22.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-19T21:24:34.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:24:34.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:37.565 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-19T21:24:38.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-19T21:24:52.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:24:52.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:52.972 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-19T21:24:53.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:24:53.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:24:53.293 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-19T21:24:53.874 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:24:53.874 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-196-generic"... 2024-09-19T21:24:53.874 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-19T21:24:53.881 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-196-generic 2024-09-19T21:24:53.881 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-19T21:24:53.881 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T21:24:53.881 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T21:24:53.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:24:53.943 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.4.0-196-generic"... 2024-09-19T21:24:53.944 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-19T21:24:53.950 INFO:teuthology.orchestra.run.smithi122.stdout:5.4.0-196-generic 2024-09-19T21:24:53.950 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-19T21:24:53.950 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T21:24:53.950 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T21:24:54.882 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T21:24:54.882 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T21:24:54.951 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-19T21:24:54.951 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T21:24:54.990 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-09-19T21:24:55.014 DEBUG:teuthology.parallel:result is None 2024-09-19T21:24:55.058 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-19T21:24:55.086 DEBUG:teuthology.parallel:result is None 2024-09-19T21:25:08.682 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T21:25:20.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:25:20.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:25:20.863 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-19T21:25:21.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:25:21.472 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-196-generic"... 2024-09-19T21:25:21.472 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-19T21:25:21.478 INFO:teuthology.orchestra.run.smithi033.stdout:5.4.0-196-generic 2024-09-19T21:25:21.478 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-19T21:25:21.479 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T21:25:21.479 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T21:25:22.479 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T21:25:22.480 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T21:25:22.567 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-09-19T21:25:22.597 DEBUG:teuthology.parallel:result is None 2024-09-19T21:25:22.597 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T21:25:22.604 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T21:25:22.604 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T21:25:22.607 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T21:25:22.609 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T21:25:22.617 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T21:25:22.700 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T21:25:22.707 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T21:25:22.707 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T21:25:22.709 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T21:25:22.713 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T21:25:22.740 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T21:25:22.745 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T21:25:22.746 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T21:25:22.748 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T21:25:22.772 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T21:25:22.774 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.784 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.786 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.797 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.800 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.811 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T21:25:22.812 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T21:25:22.820 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T21:25:22.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T21:25:22.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T21:25:22.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T21:25:22.870 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T21:25:22.882 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T21:25:22.882 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T21:25:22.885 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T21:25:22.887 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T21:25:22.894 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T21:25:22.900 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T21:25:22.951 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:25:22.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T21:25:22.976 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T21:25:22.983 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T21:25:22.990 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:25:22.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T21:25:23.015 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T21:25:23.022 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T21:25:23.070 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:25:23.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T21:25:23.099 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-09-19T21:25:23.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-09-19T21:25:23.104 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-19T21:25:23.158 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T21:25:23.166 INFO:teuthology.task.internal:Starting timer... 2024-09-19T21:25:23.166 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T21:25:23.176 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T21:25:23.184 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2024-09-19T21:25:23.185 DEBUG:teuthology.task.selinux:Excluding smithi033: OS 'ubuntu' does not support SELinux 2024-09-19T21:25:23.185 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-09-19T21:25:23.185 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T21:25:23.185 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-19T21:25:23.185 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T21:25:23.185 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T21:25:23.193 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T21:25:23.196 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T21:25:23.305 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T21:25:23.316 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T21:25:23.317 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi033.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T21:31:14.600 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-19T21:31:14.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:31:14.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:31:14.703 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-19T21:31:14.826 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-19T21:31:14.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:31:14.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:31:14.925 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-19T21:31:15.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-19T21:31:15.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:31:15.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T21:31:15.147 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-19T21:31:15.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-19T21:31:15.271 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T21:31:15.284 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T21:31:15.284 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T21:31:15.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:31:15.287 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T21:31:15.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:31:15.289 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T21:31:15.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:31:15.320 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-19T21:31:15.320 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Command line: ntpd -gq 2024-09-19T21:31:15.320 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-19T21:31:15.321 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Command line: ntpd -gq 2024-09-19T21:31:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-19T21:31:15.321 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Command line: ntpd -gq 2024-09-19T21:31:15.322 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: proto: precision = 0.116 usec (-23) 2024-09-19T21:31:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.322 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen normally on 4 lo [::1]:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-09-19T21:31:15.324 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:15 ntpd[13747]: Listening on routing socket on fd #22 for interface updates 2024-09-19T21:31:15.325 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: proto: precision = 0.103 usec (-23) 2024-09-19T21:31:15.325 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: proto: precision = 0.195 usec (-22) 2024-09-19T21:31:15.326 INFO:teuthology.orchestra.run.smithi033.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.326 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.326 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.326 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T21:31:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.327 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen normally on 3 ens1f0 172.21.15.33:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen normally on 4 lo [::1]:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:d294%4]:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:15 ntpd[13619]: Listening on routing socket on fd #22 for interface updates 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T21:31:15.329 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T21:31:15.330 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T21:31:15.330 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen normally on 3 ens1f0 172.21.15.12:123 2024-09-19T21:31:15.330 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen normally on 4 lo [::1]:123 2024-09-19T21:31:15.330 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%4]:123 2024-09-19T21:31:15.330 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:15 ntpd[13693]: Listening on routing socket on fd #22 for interface updates 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:16.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:16.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:16 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:16.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:16 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:16 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:17.321 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:17 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:17.321 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:17 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:17.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:17.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:17 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:18 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.324 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.325 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:18 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:18 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:19.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:19.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:19.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:19.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:19 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.325 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:20 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.328 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:20 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:20.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:20.331 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:20 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:21.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:21.322 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:21.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:21.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:21.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:21.323 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:21 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.322 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:19 Sep 21:31:22 ntpd[13747]: ntpd: time slew -0.000097 s 2024-09-19T21:31:22.323 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.000097s 2024-09-19T21:31:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T21:31:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:19 Sep 21:31:22 ntpd[13747]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.326 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stdout:19 Sep 21:31:22 ntpd[13693]: ntpd: time slew +0.003090 s 2024-09-19T21:31:22.327 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.003090s 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.328 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stdout:19 Sep 21:31:22 ntpd[13619]: ntpd: time slew +0.002475 s 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi033.stdout:ntpd: time slew +0.002475s 2024-09-19T21:31:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T21:31:22.330 INFO:teuthology.orchestra.run.smithi012.stderr:19 Sep 21:31:22 ntpd[13693]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T21:31:22.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T21:31:22.330 INFO:teuthology.orchestra.run.smithi033.stderr:19 Sep 21:31:22 ntpd[13619]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T21:31:22.386 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:31:22.387 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-19T21:31:22.387 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.387 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.387 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.387 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:31:22.403 INFO:teuthology.run_tasks:Running task install... 2024-09-19T21:31:22.417 DEBUG:teuthology.task.install:project ceph 2024-09-19T21:31:22.417 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-19T21:31:22.417 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-19T21:31:22.417 INFO:teuthology.task.install:Using flavor: default 2024-09-19T21:31:22.423 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T21:31:22.423 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-19T21:31:22.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2024-09-19T21:31:22.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-key list | grep Ceph 2024-09-19T21:31:22.424 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-09-19T21:31:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-19T21:31:22.543 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-19T21:31:22.556 INFO:teuthology.orchestra.run.smithi033.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-19T21:31:22.579 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T21:31:22.579 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T21:31:22.580 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-19T21:31:22.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:31:22.603 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T21:31:22.603 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T21:31:22.604 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-19T21:31:22.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:31:22.619 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T21:31:22.619 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T21:31:22.620 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-19T21:31:22.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:31:22.722 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default/ 2024-09-19T21:31:22.722 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:22.755 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default/ 2024-09-19T21:31:22.755 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:22.756 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default/ 2024-09-19T21:31:22.756 INFO:teuthology.task.install.deb:Package version is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:22.832 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:31:22.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T21:31:22.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-19T21:31:22.870 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:31:22.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T21:31:22.874 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:31:22.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T21:31:22.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-09-19T21:31:22.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-19T21:31:23.048 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:31:23.048 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:31:23.049 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:31:23.049 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-19T21:31:23.053 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:31:23.071 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:31:23.084 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal InRelease 2024-09-19T21:31:23.086 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal InRelease 2024-09-19T21:31:23.088 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal InRelease 2024-09-19T21:31:23.105 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-19T21:31:23.108 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-19T21:31:23.109 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-19T21:31:23.128 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release.gpg 2024-09-19T21:31:23.131 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release.gpg 2024-09-19T21:31:23.132 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal Release.gpg 2024-09-19T21:31:23.150 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-19T21:31:23.152 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main i386 Packages [3,426 B] 2024-09-19T21:31:23.153 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-19T21:31:23.164 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:31:23.165 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:6 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-19T21:31:23.181 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main i386 Packages [3,426 B] 2024-09-19T21:31:23.185 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-19T21:31:23.185 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main i386 Packages [3,426 B] 2024-09-19T21:31:23.245 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:31:23.246 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-19T21:31:23.327 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:31:23.333 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-19T21:31:23.710 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 28.7 kB in 1s (44.2 kB/s) 2024-09-19T21:31:23.717 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 28.7 kB in 1s (43.2 kB/s) 2024-09-19T21:31:23.726 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 28.7 kB in 1s (42.7 kB/s) 2024-09-19T21:31:24.870 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:31:24.897 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1focal cephadm=18.2.4-855-g19e2a2be-1focal ceph-mds=18.2.4-855-g19e2a2be-1focal ceph-mgr=18.2.4-855-g19e2a2be-1focal ceph-common=18.2.4-855-g19e2a2be-1focal ceph-fuse=18.2.4-855-g19e2a2be-1focal ceph-test=18.2.4-855-g19e2a2be-1focal ceph-volume=18.2.4-855-g19e2a2be-1focal radosgw=18.2.4-855-g19e2a2be-1focal python3-rados=18.2.4-855-g19e2a2be-1focal python3-rgw=18.2.4-855-g19e2a2be-1focal python3-cephfs=18.2.4-855-g19e2a2be-1focal python3-rbd=18.2.4-855-g19e2a2be-1focal libcephfs2=18.2.4-855-g19e2a2be-1focal libcephfs-dev=18.2.4-855-g19e2a2be-1focal librados2=18.2.4-855-g19e2a2be-1focal librbd1=18.2.4-855-g19e2a2be-1focal rbd-fuse=18.2.4-855-g19e2a2be-1focal rbd-nbd=18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:24.926 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:31:24.937 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:31:24.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1focal cephadm=18.2.4-855-g19e2a2be-1focal ceph-mds=18.2.4-855-g19e2a2be-1focal ceph-mgr=18.2.4-855-g19e2a2be-1focal ceph-common=18.2.4-855-g19e2a2be-1focal ceph-fuse=18.2.4-855-g19e2a2be-1focal ceph-test=18.2.4-855-g19e2a2be-1focal ceph-volume=18.2.4-855-g19e2a2be-1focal radosgw=18.2.4-855-g19e2a2be-1focal python3-rados=18.2.4-855-g19e2a2be-1focal python3-rgw=18.2.4-855-g19e2a2be-1focal python3-cephfs=18.2.4-855-g19e2a2be-1focal python3-rbd=18.2.4-855-g19e2a2be-1focal libcephfs2=18.2.4-855-g19e2a2be-1focal libcephfs-dev=18.2.4-855-g19e2a2be-1focal librados2=18.2.4-855-g19e2a2be-1focal librbd1=18.2.4-855-g19e2a2be-1focal rbd-fuse=18.2.4-855-g19e2a2be-1focal rbd-nbd=18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:24.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-855-g19e2a2be-1focal cephadm=18.2.4-855-g19e2a2be-1focal ceph-mds=18.2.4-855-g19e2a2be-1focal ceph-mgr=18.2.4-855-g19e2a2be-1focal ceph-common=18.2.4-855-g19e2a2be-1focal ceph-fuse=18.2.4-855-g19e2a2be-1focal ceph-test=18.2.4-855-g19e2a2be-1focal ceph-volume=18.2.4-855-g19e2a2be-1focal radosgw=18.2.4-855-g19e2a2be-1focal python3-rados=18.2.4-855-g19e2a2be-1focal python3-rgw=18.2.4-855-g19e2a2be-1focal python3-cephfs=18.2.4-855-g19e2a2be-1focal python3-rbd=18.2.4-855-g19e2a2be-1focal libcephfs2=18.2.4-855-g19e2a2be-1focal libcephfs-dev=18.2.4-855-g19e2a2be-1focal librados2=18.2.4-855-g19e2a2be-1focal librbd1=18.2.4-855-g19e2a2be-1focal rbd-fuse=18.2.4-855-g19e2a2be-1focal rbd-nbd=18.2.4-855-g19e2a2be-1focal 2024-09-19T21:31:24.965 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-19T21:31:25.029 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-19T21:31:25.036 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-09-19T21:31:25.185 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-19T21:31:25.186 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-19T21:31:25.230 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-19T21:31:25.231 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-19T21:31:25.262 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-09-19T21:31:25.262 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-09-19T21:31:25.383 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T21:31:25.384 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-19T21:31:25.384 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T21:31:25.384 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-19T21:31:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T21:31:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-19T21:31:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-19T21:31:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-19T21:31:25.385 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-19T21:31:25.387 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-19T21:31:25.388 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-09-19T21:31:25.435 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T21:31:25.436 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-19T21:31:25.436 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-19T21:31:25.437 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.438 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-19T21:31:25.439 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-19T21:31:25.440 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2024-09-19T21:31:25.477 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T21:31:25.478 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-19T21:31:25.478 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-19T21:31:25.479 INFO:teuthology.orchestra.run.smithi033.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-19T21:31:25.480 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-19T21:31:25.480 INFO:teuthology.orchestra.run.smithi033.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-19T21:31:25.481 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-19T21:31:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-09-19T21:31:25.483 INFO:teuthology.orchestra.run.smithi033.stdout: btrfs-tools 2024-09-19T21:31:25.510 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-19T21:31:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T21:31:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T21:31:25.510 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T21:31:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-19T21:31:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-19T21:31:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-19T21:31:25.511 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-19T21:31:25.512 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse 2024-09-19T21:31:25.513 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd socat xmlstarlet 2024-09-19T21:31:25.514 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-19T21:31:25.514 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 librbd1 2024-09-19T21:31:25.565 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-19T21:31:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T21:31:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T21:31:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T21:31:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-19T21:31:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-19T21:31:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-19T21:31:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd socat xmlstarlet 2024-09-19T21:31:25.568 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-09-19T21:31:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 librbd1 2024-09-19T21:31:25.606 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-19T21:31:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-19T21:31:25.609 INFO:teuthology.orchestra.run.smithi033.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-19T21:31:25.609 INFO:teuthology.orchestra.run.smithi033.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-19T21:31:25.609 INFO:teuthology.orchestra.run.smithi033.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T21:31:25.610 INFO:teuthology.orchestra.run.smithi033.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-19T21:31:25.611 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-19T21:31:25.611 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse 2024-09-19T21:31:25.611 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd socat xmlstarlet 2024-09-19T21:31:25.611 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-09-19T21:31:25.611 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 librbd1 2024-09-19T21:31:25.633 INFO:teuthology.orchestra.run.smithi012.stdout:2 upgraded, 111 newly installed, 0 to remove and 297 not upgraded. 2024-09-19T21:31:25.633 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 199 MB of archives. 2024-09-19T21:31:25.633 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 978 MB of additional disk space will be used. 2024-09-19T21:31:25.633 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-19T21:31:25.662 INFO:teuthology.orchestra.run.smithi122.stdout:2 upgraded, 111 newly installed, 0 to remove and 297 not upgraded. 2024-09-19T21:31:25.662 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 199 MB of archives. 2024-09-19T21:31:25.662 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 978 MB of additional disk space will be used. 2024-09-19T21:31:25.663 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1focal [3,453 kB] 2024-09-19T21:31:25.675 INFO:teuthology.orchestra.run.smithi033.stdout:2 upgraded, 111 newly installed, 0 to remove and 297 not upgraded. 2024-09-19T21:31:25.675 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 199 MB of archives. 2024-09-19T21:31:25.675 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 978 MB of additional disk space will be used. 2024-09-19T21:31:25.675 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-19T21:31:25.707 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-19T21:31:25.717 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1focal [3,453 kB] 2024-09-19T21:31:25.718 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-19T21:31:25.722 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-19T21:31:25.740 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-19T21:31:25.759 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-19T21:31:25.759 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-855-g19e2a2be-1focal [3,453 kB] 2024-09-19T21:31:25.762 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-19T21:31:25.780 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-19T21:31:25.795 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-19T21:31:25.798 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-19T21:31:25.805 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-19T21:31:25.807 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-19T21:31:25.807 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-19T21:31:25.809 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-19T21:31:25.809 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-19T21:31:25.812 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-19T21:31:25.812 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-19T21:31:25.814 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-19T21:31:25.831 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-19T21:31:25.832 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-19T21:31:25.832 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-19T21:31:25.832 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T21:31:25.833 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-19T21:31:25.833 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-19T21:31:25.834 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-19T21:31:25.835 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-19T21:31:25.836 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-19T21:31:25.836 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-19T21:31:25.843 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-19T21:31:25.845 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-19T21:31:25.845 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-19T21:31:25.846 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-19T21:31:25.846 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-19T21:31:25.848 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-19T21:31:25.849 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-19T21:31:25.850 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-19T21:31:25.850 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-19T21:31:25.852 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-19T21:31:25.866 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-19T21:31:25.869 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-19T21:31:25.869 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-19T21:31:25.870 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-19T21:31:25.871 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T21:31:25.871 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-19T21:31:25.872 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-19T21:31:25.873 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-19T21:31:25.874 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-19T21:31:25.875 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-19T21:31:25.875 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-19T21:31:25.875 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-19T21:31:25.876 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-19T21:31:25.877 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-19T21:31:25.880 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-19T21:31:25.882 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-19T21:31:25.883 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-19T21:31:25.885 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1focal [4,044 kB] 2024-09-19T21:31:25.885 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-19T21:31:25.886 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-19T21:31:25.886 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-19T21:31:25.894 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-19T21:31:25.900 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-19T21:31:25.900 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-19T21:31:25.901 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-19T21:31:25.901 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-19T21:31:25.901 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-19T21:31:25.902 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-19T21:31:25.902 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-19T21:31:25.903 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-19T21:31:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-19T21:31:25.903 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-19T21:31:25.904 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-19T21:31:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-19T21:31:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-19T21:31:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-19T21:31:25.918 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-19T21:31:25.919 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-19T21:31:25.919 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-19T21:31:25.921 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-19T21:31:25.922 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T21:31:25.922 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-19T21:31:25.924 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-19T21:31:25.925 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-19T21:31:25.926 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-19T21:31:25.933 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1focal [4,044 kB] 2024-09-19T21:31:25.935 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-19T21:31:25.938 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-19T21:31:25.938 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-19T21:31:25.939 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-19T21:31:25.939 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-19T21:31:25.940 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-19T21:31:25.940 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-19T21:31:25.940 INFO:teuthology.orchestra.run.smithi033.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-19T21:31:25.941 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-19T21:31:25.941 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-19T21:31:25.942 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-19T21:31:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T21:31:25.955 INFO:teuthology.orchestra.run.smithi033.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-19T21:31:25.957 INFO:teuthology.orchestra.run.smithi033.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-19T21:31:25.959 INFO:teuthology.orchestra.run.smithi033.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-19T21:31:25.960 INFO:teuthology.orchestra.run.smithi033.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-19T21:31:25.972 INFO:teuthology.orchestra.run.smithi033.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-19T21:31:25.973 INFO:teuthology.orchestra.run.smithi033.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-19T21:31:25.989 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1focal [746 kB] 2024-09-19T21:31:26.001 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1focal [388 kB] 2024-09-19T21:31:26.025 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-19T21:31:26.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-19T21:31:26.042 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-19T21:31:26.044 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1focal [31.9 kB] 2024-09-19T21:31:26.047 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1focal [201 kB] 2024-09-19T21:31:26.051 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1focal [62.5 kB] 2024-09-19T21:31:26.053 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1focal [367 kB] 2024-09-19T21:31:26.054 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-19T21:31:26.054 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-19T21:31:26.054 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-19T21:31:26.057 INFO:teuthology.orchestra.run.smithi033.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-19T21:31:26.058 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1focal [6,627 kB] 2024-09-19T21:31:26.075 INFO:teuthology.orchestra.run.smithi033.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-19T21:31:26.077 INFO:teuthology.orchestra.run.smithi033.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-19T21:31:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-19T21:31:26.104 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-19T21:31:26.105 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-19T21:31:26.106 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-19T21:31:26.107 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-19T21:31:26.107 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-19T21:31:26.109 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-19T21:31:26.109 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-19T21:31:26.110 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-19T21:31:26.115 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-19T21:31:26.121 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-19T21:31:26.122 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-19T21:31:26.136 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-19T21:31:26.137 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-19T21:31:26.141 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-19T21:31:26.145 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-19T21:31:26.145 INFO:teuthology.orchestra.run.smithi033.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-19T21:31:26.146 INFO:teuthology.orchestra.run.smithi033.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-19T21:31:26.147 INFO:teuthology.orchestra.run.smithi033.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-19T21:31:26.148 INFO:teuthology.orchestra.run.smithi033.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-19T21:31:26.149 INFO:teuthology.orchestra.run.smithi033.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-19T21:31:26.150 INFO:teuthology.orchestra.run.smithi033.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-19T21:31:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-19T21:31:26.152 INFO:teuthology.orchestra.run.smithi033.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-19T21:31:26.160 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1focal [746 kB] 2024-09-19T21:31:26.161 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-19T21:31:26.163 INFO:teuthology.orchestra.run.smithi033.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-19T21:31:26.164 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-19T21:31:26.166 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-19T21:31:26.170 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1focal [388 kB] 2024-09-19T21:31:26.170 INFO:teuthology.orchestra.run.smithi033.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-19T21:31:26.173 INFO:teuthology.orchestra.run.smithi033.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-19T21:31:26.173 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1focal [31.9 kB] 2024-09-19T21:31:26.174 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1focal [201 kB] 2024-09-19T21:31:26.175 INFO:teuthology.orchestra.run.smithi033.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-19T21:31:26.175 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1focal [62.5 kB] 2024-09-19T21:31:26.176 INFO:teuthology.orchestra.run.smithi033.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-19T21:31:26.176 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1focal [367 kB] 2024-09-19T21:31:26.179 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1focal [6,627 kB] 2024-09-19T21:31:26.179 INFO:teuthology.orchestra.run.smithi033.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-19T21:31:26.183 INFO:teuthology.orchestra.run.smithi033.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-19T21:31:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-19T21:31:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-19T21:31:26.202 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-19T21:31:26.202 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-19T21:31:26.203 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-19T21:31:26.203 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-19T21:31:26.203 INFO:teuthology.orchestra.run.smithi033.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-19T21:31:26.205 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-19T21:31:26.215 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T21:31:26.215 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-19T21:31:26.279 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-19T21:31:26.280 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-19T21:31:26.281 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-19T21:31:26.281 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1focal [113 kB] 2024-09-19T21:31:26.283 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1focal [478 kB] 2024-09-19T21:31:26.320 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1focal [25.2 MB] 2024-09-19T21:31:26.355 INFO:teuthology.orchestra.run.smithi033.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-19T21:31:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-19T21:31:26.372 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1focal [113 kB] 2024-09-19T21:31:26.373 INFO:teuthology.orchestra.run.smithi033.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-19T21:31:26.374 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1focal [478 kB] 2024-09-19T21:31:26.374 INFO:teuthology.orchestra.run.smithi033.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-19T21:31:26.390 INFO:teuthology.orchestra.run.smithi033.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T21:31:26.390 INFO:teuthology.orchestra.run.smithi033.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-19T21:31:26.392 INFO:teuthology.orchestra.run.smithi033.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-19T21:31:26.392 INFO:teuthology.orchestra.run.smithi033.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-19T21:31:26.392 INFO:teuthology.orchestra.run.smithi033.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-19T21:31:26.418 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1focal [25.2 MB] 2024-09-19T21:31:26.447 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-19T21:31:26.456 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-19T21:31:26.484 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-19T21:31:26.493 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-19T21:31:26.493 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-19T21:31:26.497 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-19T21:31:26.498 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-19T21:31:26.510 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-19T21:31:26.512 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-19T21:31:26.529 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-19T21:31:26.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-19T21:31:26.600 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-19T21:31:26.601 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-19T21:31:26.604 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T21:31:26.604 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-19T21:31:26.605 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-19T21:31:26.607 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-19T21:31:26.608 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-19T21:31:26.610 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-19T21:31:26.656 INFO:teuthology.orchestra.run.smithi033.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-855-g19e2a2be-1focal [4,044 kB] 2024-09-19T21:31:26.691 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-19T21:31:26.692 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-19T21:31:26.701 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-19T21:31:26.702 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-19T21:31:26.703 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-19T21:31:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-19T21:31:26.705 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-19T21:31:26.707 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-19T21:31:26.773 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-19T21:31:26.778 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-19T21:31:26.854 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-19T21:31:26.857 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-19T21:31:26.861 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-19T21:31:26.861 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-19T21:31:26.862 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-19T21:31:26.863 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-19T21:31:26.865 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-19T21:31:26.866 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-19T21:31:26.936 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-19T21:31:26.936 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-19T21:31:26.949 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T21:31:26.949 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-19T21:31:26.951 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-19T21:31:26.953 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-19T21:31:27.018 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-19T21:31:27.018 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-19T21:31:27.019 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-19T21:31:27.172 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1focal [9,605 kB] 2024-09-19T21:31:27.300 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-19T21:31:27.374 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-19T21:31:27.377 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-19T21:31:27.425 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1focal [238 kB] 2024-09-19T21:31:27.427 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1focal [148 kB] 2024-09-19T21:31:27.447 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-19T21:31:27.447 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-19T21:31:27.447 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1focal [1,574 kB] 2024-09-19T21:31:27.448 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-19T21:31:27.475 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1focal [9,605 kB] 2024-09-19T21:31:27.506 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-19T21:31:27.506 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1focal [6,640 kB] 2024-09-19T21:31:27.507 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-19T21:31:27.508 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-19T21:31:27.508 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-19T21:31:27.510 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-19T21:31:27.511 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-19T21:31:27.546 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-19T21:31:27.551 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-19T21:31:27.627 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-19T21:31:27.631 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-19T21:31:27.632 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-19T21:31:27.637 INFO:teuthology.orchestra.run.smithi033.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-855-g19e2a2be-1focal [746 kB] 2024-09-19T21:31:27.639 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-19T21:31:27.642 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-19T21:31:27.702 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1focal [24.5 MB] 2024-09-19T21:31:27.712 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-19T21:31:27.718 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-19T21:31:27.723 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-19T21:31:27.759 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1focal [238 kB] 2024-09-19T21:31:27.762 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1focal [148 kB] 2024-09-19T21:31:27.805 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1focal [1,574 kB] 2024-09-19T21:31:27.831 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1focal [6,640 kB] 2024-09-19T21:31:27.841 INFO:teuthology.orchestra.run.smithi033.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-855-g19e2a2be-1focal [388 kB] 2024-09-19T21:31:27.885 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-19T21:31:27.887 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-19T21:31:27.887 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-19T21:31:27.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-19T21:31:27.913 INFO:teuthology.orchestra.run.smithi033.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-855-g19e2a2be-1focal [31.9 kB] 2024-09-19T21:31:27.915 INFO:teuthology.orchestra.run.smithi033.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-855-g19e2a2be-1focal [201 kB] 2024-09-19T21:31:27.926 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T21:31:27.927 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-19T21:31:27.929 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-19T21:31:27.929 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-19T21:31:27.931 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-19T21:31:27.982 INFO:teuthology.orchestra.run.smithi033.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-855-g19e2a2be-1focal [62.5 kB] 2024-09-19T21:31:27.984 INFO:teuthology.orchestra.run.smithi033.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-855-g19e2a2be-1focal [367 kB] 2024-09-19T21:31:28.056 INFO:teuthology.orchestra.run.smithi033.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-855-g19e2a2be-1focal [6,627 kB] 2024-09-19T21:31:28.102 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1focal [24.5 MB] 2024-09-19T21:31:28.571 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1focal [13.7 kB] 2024-09-19T21:31:28.625 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1focal [856 kB] 2024-09-19T21:31:28.640 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1focal [2,295 kB] 2024-09-19T21:31:28.682 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1focal [239 kB] 2024-09-19T21:31:28.684 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1focal [143 kB] 2024-09-19T21:31:28.687 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1focal [7,015 kB] 2024-09-19T21:31:28.886 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1focal [7,893 kB] 2024-09-19T21:31:29.076 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1focal [13.7 kB] 2024-09-19T21:31:29.133 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1focal [856 kB] 2024-09-19T21:31:29.133 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1focal [13.9 kB] 2024-09-19T21:31:29.134 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1focal [46.9 MB] 2024-09-19T21:31:29.145 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1focal [2,295 kB] 2024-09-19T21:31:29.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1focal [239 kB] 2024-09-19T21:31:29.164 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1focal [143 kB] 2024-09-19T21:31:29.165 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1focal [7,015 kB] 2024-09-19T21:31:29.441 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1focal [7,893 kB] 2024-09-19T21:31:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-855-g19e2a2be-1focal [113 kB] 2024-09-19T21:31:29.686 INFO:teuthology.orchestra.run.smithi033.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-855-g19e2a2be-1focal [478 kB] 2024-09-19T21:31:29.744 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1focal [13.9 kB] 2024-09-19T21:31:29.744 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1focal [46.9 MB] 2024-09-19T21:31:29.767 INFO:teuthology.orchestra.run.smithi033.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-855-g19e2a2be-1focal [25.2 MB] 2024-09-19T21:31:30.901 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1focal [120 kB] 2024-09-19T21:31:30.903 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1focal [37.4 kB] 2024-09-19T21:31:30.905 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1focal [13.5 MB] 2024-09-19T21:31:31.357 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1focal [108 kB] 2024-09-19T21:31:31.358 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-nbd amd64 18.2.4-855-g19e2a2be-1focal [189 kB] 2024-09-19T21:31:31.519 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1focal [120 kB] 2024-09-19T21:31:31.522 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1focal [37.4 kB] 2024-09-19T21:31:31.523 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1focal [13.5 MB] 2024-09-19T21:31:31.720 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1focal [108 kB] 2024-09-19T21:31:31.721 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-nbd amd64 18.2.4-855-g19e2a2be-1focal [189 kB] 2024-09-19T21:31:32.010 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 199 MB in 6s (34.1 MB/s) 2024-09-19T21:31:32.182 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-19T21:31:32.240 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-19T21:31:32.244 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-19T21:31:32.331 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:31:32.460 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 199 MB in 6s (32.0 MB/s) 2024-09-19T21:31:32.636 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-19T21:31:32.651 INFO:teuthology.orchestra.run.smithi033.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-855-g19e2a2be-1focal [9,605 kB] 2024-09-19T21:31:32.692 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-19T21:31:32.696 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-19T21:31:32.718 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T21:31:32.735 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-19T21:31:32.776 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:31:32.776 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:31:33.154 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T21:31:33.172 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-19T21:31:33.172 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T21:31:33.192 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-19T21:31:33.230 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:31:33.232 INFO:teuthology.orchestra.run.smithi033.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-855-g19e2a2be-1focal [238 kB] 2024-09-19T21:31:33.238 INFO:teuthology.orchestra.run.smithi033.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-855-g19e2a2be-1focal [148 kB] 2024-09-19T21:31:33.246 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:31:33.252 INFO:teuthology.orchestra.run.smithi033.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-855-g19e2a2be-1focal [1,574 kB] 2024-09-19T21:31:33.343 INFO:teuthology.orchestra.run.smithi033.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-855-g19e2a2be-1focal [6,640 kB] 2024-09-19T21:31:33.558 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T21:31:33.576 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-19T21:31:33.607 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:31:33.741 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T21:31:33.751 INFO:teuthology.orchestra.run.smithi033.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-855-g19e2a2be-1focal [24.5 MB] 2024-09-19T21:31:33.757 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:33.810 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:34.102 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T21:31:34.118 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:34.171 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:34.352 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T21:31:34.371 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:34.410 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:34.689 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T21:31:34.706 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:34.747 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:34.806 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T21:31:34.823 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:34.889 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:35.058 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T21:31:35.077 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:35.116 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:35.249 INFO:teuthology.orchestra.run.smithi033.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-855-g19e2a2be-1focal [13.7 kB] 2024-09-19T21:31:35.293 INFO:teuthology.orchestra.run.smithi033.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-855-g19e2a2be-1focal [856 kB] 2024-09-19T21:31:35.300 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-19T21:31:35.308 INFO:teuthology.orchestra.run.smithi033.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-855-g19e2a2be-1focal [2,295 kB] 2024-09-19T21:31:35.316 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-19T21:31:35.358 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:31:35.384 INFO:teuthology.orchestra.run.smithi033.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-855-g19e2a2be-1focal [239 kB] 2024-09-19T21:31:35.400 INFO:teuthology.orchestra.run.smithi033.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-855-g19e2a2be-1focal [143 kB] 2024-09-19T21:31:35.403 INFO:teuthology.orchestra.run.smithi033.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-855-g19e2a2be-1focal [7,015 kB] 2024-09-19T21:31:35.503 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-19T21:31:35.519 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-19T21:31:35.569 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:31:35.744 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-librbd1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:35.758 INFO:teuthology.orchestra.run.smithi033.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-855-g19e2a2be-1focal [7,893 kB] 2024-09-19T21:31:35.904 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:35.947 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-librbd1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:36.098 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:36.143 INFO:teuthology.orchestra.run.smithi033.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-855-g19e2a2be-1focal [13.9 kB] 2024-09-19T21:31:36.143 INFO:teuthology.orchestra.run.smithi033.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-855-g19e2a2be-1focal [46.9 MB] 2024-09-19T21:31:36.624 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:36.750 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:36.761 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:36.885 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:37.512 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T21:31:37.528 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:37.570 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:37.647 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T21:31:37.663 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:37.731 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:37.932 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-09-19T21:31:37.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-python3-rados_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:37.989 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.058 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2024-09-19T21:31:38.074 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-python3-rados_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:38.116 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.376 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T21:31:38.392 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:38.444 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T21:31:38.459 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.514 INFO:teuthology.orchestra.run.smithi033.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-855-g19e2a2be-1focal [120 kB] 2024-09-19T21:31:38.516 INFO:teuthology.orchestra.run.smithi033.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-855-g19e2a2be-1focal [37.4 kB] 2024-09-19T21:31:38.517 INFO:teuthology.orchestra.run.smithi033.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-855-g19e2a2be-1focal [13.5 MB] 2024-09-19T21:31:38.711 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T21:31:38.728 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:38.746 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T21:31:38.762 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:38.787 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.805 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:38.971 INFO:teuthology.orchestra.run.smithi033.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-855-g19e2a2be-1focal [108 kB] 2024-09-19T21:31:39.057 INFO:teuthology.orchestra.run.smithi033.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/ubuntu/focal/flavors/default focal/main amd64 rbd-nbd amd64 18.2.4-855-g19e2a2be-1focal [189 kB] 2024-09-19T21:31:39.107 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T21:31:39.125 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:39.156 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T21:31:39.165 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:39.172 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:39.214 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:39.502 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T21:31:39.521 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-19T21:31:39.560 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-19T21:31:39.576 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T21:31:39.592 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-19T21:31:39.634 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-19T21:31:39.642 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 199 MB in 13s (14.9 MB/s) 2024-09-19T21:31:39.821 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T21:31:39.828 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-19T21:31:39.837 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:39.884 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-19T21:31:39.888 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-19T21:31:39.904 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:39.911 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T21:31:39.928 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:39.960 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:31:39.970 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:40.248 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-19T21:31:40.268 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-19T21:31:40.307 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:31:40.330 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T21:31:40.349 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-19T21:31:40.364 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-19T21:31:40.380 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-19T21:31:40.413 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:31:40.423 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:31:40.668 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-19T21:31:40.687 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-19T21:31:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:31:40.791 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T21:31:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-19T21:31:40.826 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-19T21:31:40.845 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-19T21:31:40.882 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:31:40.884 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:31:41.088 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T21:31:41.104 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-19T21:31:41.146 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:31:41.245 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T21:31:41.265 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-19T21:31:41.303 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:31:41.411 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T21:31:41.427 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:41.524 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2024-09-19T21:31:41.541 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:41.582 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:41.690 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-09-19T21:31:41.707 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:41.748 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:42.014 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T21:31:42.030 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:42.106 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:42.500 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T21:31:42.509 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T21:31:42.516 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-19T21:31:42.526 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:42.567 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:42.583 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:31:42.659 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T21:31:42.675 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:42.717 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:42.929 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0. 2024-09-19T21:31:42.946 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-19T21:31:42.987 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-19T21:31:43.039 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-19T21:31:43.057 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-19T21:31:43.195 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:31:43.279 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0. 2024-09-19T21:31:43.295 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-19T21:31:43.337 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-19T21:31:43.340 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T21:31:43.360 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:43.398 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:43.598 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../007-librbd1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:43.682 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T21:31:43.697 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:43.700 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librbd1 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:43.740 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:43.868 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2024-09-19T21:31:43.888 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:43.926 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:44.235 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-09-19T21:31:44.251 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:44.293 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:44.456 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../008-librados2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:44.586 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librados2 (18.2.4-855-g19e2a2be-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-19T21:31:45.298 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T21:31:45.317 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:45.356 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:45.684 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rados. 2024-09-19T21:31:45.703 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../010-python3-rados_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:45.775 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:46.128 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T21:31:46.144 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:46.186 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:46.398 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2024-09-19T21:31:46.417 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:46.467 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:46.506 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T21:31:46.525 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:46.564 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:46.632 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-09-19T21:31:46.650 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:46.692 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:46.925 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T21:31:46.944 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:47.016 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:47.386 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T21:31:47.402 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-19T21:31:47.444 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-19T21:31:47.763 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T21:31:47.769 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T21:31:47.780 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:47.789 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-19T21:31:47.822 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:47.827 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:31:47.945 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T21:31:47.963 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-19T21:31:48.003 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:31:48.197 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T21:31:48.216 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-19T21:31:48.218 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-19T21:31:48.235 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-19T21:31:48.255 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-19T21:31:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:31:48.398 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T21:31:48.417 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-19T21:31:48.457 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-19T21:31:48.600 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T21:31:48.616 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-19T21:31:48.658 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-19T21:31:48.669 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-19T21:31:48.686 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-19T21:31:48.745 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:31:48.784 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T21:31:48.802 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-19T21:31:48.842 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-19T21:31:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T21:31:48.996 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-19T21:31:49.036 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:31:49.114 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T21:31:49.130 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-19T21:31:49.162 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T21:31:49.182 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-19T21:31:49.197 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:31:49.220 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:31:49.322 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2024-09-19T21:31:49.341 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-19T21:31:49.371 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:31:49.549 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-09-19T21:31:49.564 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-19T21:31:49.566 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librgw2. 2024-09-19T21:31:49.583 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../019-librgw2_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:49.606 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:31:49.624 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:49.666 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T21:31:49.685 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T21:31:49.749 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:31:49.968 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T21:31:49.984 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T21:31:50.011 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T21:31:50.027 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-19T21:31:50.043 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:31:50.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:31:50.338 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2024-09-19T21:31:50.357 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-19T21:31:50.363 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T21:31:50.379 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-19T21:31:50.388 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:31:50.420 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:31:50.544 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T21:31:50.563 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:50.602 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:50.699 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-19T21:31:50.718 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-19T21:31:50.757 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-09-19T21:31:50.757 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:31:50.777 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-19T21:31:50.815 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:31:50.947 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liboath0. 2024-09-19T21:31:50.964 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-19T21:31:51.013 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-19T21:31:51.118 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bs4. 2024-09-19T21:31:51.137 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-19T21:31:51.160 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-19T21:31:51.176 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-19T21:31:51.176 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-19T21:31:51.235 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:31:51.358 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T21:31:51.374 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:51.415 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:51.546 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bs4. 2024-09-19T21:31:51.563 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-19T21:31:51.571 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T21:31:51.588 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-19T21:31:51.633 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:31:51.637 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-19T21:31:51.894 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-common. 2024-09-19T21:31:51.913 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../023-ceph-common_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:51.983 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T21:31:51.985 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:51.999 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-19T21:31:52.016 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T21:31:52.032 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-19T21:31:52.041 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-19T21:31:52.077 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:31:52.363 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2024-09-19T21:31:52.382 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-19T21:31:52.417 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T21:31:52.427 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:31:52.435 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-19T21:31:52.475 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-19T21:31:52.730 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T21:31:52.747 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-19T21:31:52.780 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:31:52.812 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-09-19T21:31:52.828 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-19T21:31:52.903 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:31:53.097 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T21:31:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-19T21:31:53.147 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:31:53.249 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T21:31:53.266 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-19T21:31:53.307 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:31:53.483 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T21:31:53.499 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-19T21:31:53.541 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-19T21:31:53.651 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T21:31:53.670 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-19T21:31:53.709 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:31:53.861 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T21:31:53.881 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-19T21:31:53.919 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:31:54.063 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T21:31:54.079 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-19T21:31:54.121 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-19T21:31:54.282 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T21:31:54.299 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-19T21:31:54.340 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:31:54.488 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T21:31:54.506 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-19T21:31:54.541 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-base. 2024-09-19T21:31:54.557 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:31:54.558 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../024-ceph-base_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:54.610 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:54.635 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T21:31:54.652 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:54.693 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:54.929 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T21:31:54.944 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-19T21:31:54.987 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:31:55.067 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T21:31:55.085 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:55.125 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:55.325 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T21:31:55.341 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:31:55.384 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:55.486 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T21:31:55.503 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-19T21:31:55.545 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:31:55.818 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T21:31:55.826 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cryptography. 2024-09-19T21:31:55.833 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:55.843 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-19T21:31:55.876 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:55.892 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:31:55.945 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T21:31:55.962 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-19T21:31:55.995 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:31:56.219 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T21:31:56.236 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-19T21:31:56.262 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T21:31:56.277 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:31:56.279 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-19T21:31:56.337 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-19T21:31:56.424 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T21:31:56.439 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-19T21:31:56.481 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-19T21:31:56.614 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cryptography. 2024-09-19T21:31:56.631 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-19T21:31:56.672 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:31:56.681 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T21:31:56.699 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:56.756 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:56.760 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T21:31:56.776 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-19T21:31:56.809 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-19T21:31:57.043 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T21:31:57.058 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-19T21:31:57.134 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-19T21:31:57.162 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T21:31:57.180 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-19T21:31:57.221 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:31:57.267 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T21:31:57.283 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:57.504 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T21:31:57.520 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:57.557 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-mako. 2024-09-19T21:31:57.562 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:57.573 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-19T21:31:57.639 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:31:57.767 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:57.993 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T21:31:58.012 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T21:31:58.051 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:31:58.099 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T21:31:58.115 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:58.157 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:58.413 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T21:31:58.430 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-19T21:31:58.470 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:31:58.779 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T21:31:58.795 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:58.837 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:31:59.557 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webob. 2024-09-19T21:31:59.577 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-19T21:31:59.615 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:31:59.861 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T21:31:59.877 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:31:59.920 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:00.018 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-19T21:32:00.034 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-19T21:32:00.068 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:32:00.354 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bs4. 2024-09-19T21:32:00.370 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-19T21:32:00.403 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-19T21:32:00.790 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T21:32:00.806 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-19T21:32:00.851 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:32:01.091 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2024-09-19T21:32:01.107 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:01.149 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:01.160 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T21:32:01.180 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-19T21:32:01.251 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-19T21:32:01.510 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T21:32:01.527 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:01.568 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:01.579 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-paste. 2024-09-19T21:32:01.595 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-19T21:32:01.637 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:32:01.980 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T21:32:01.998 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:02.005 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T21:32:02.024 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-19T21:32:02.038 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:02.063 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:32:02.148 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-09-19T21:32:02.164 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:02.390 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T21:32:02.407 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-19T21:32:02.490 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:32:02.567 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T21:32:02.574 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2024-09-19T21:32:02.583 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:02.594 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-cephadm_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:02.625 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:02.632 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:02.802 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T21:32:02.821 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-19T21:32:02.852 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-19T21:32:02.977 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T21:32:02.997 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-19T21:32:03.035 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-19T21:32:03.070 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T21:32:03.087 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:03.128 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:03.204 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T21:32:03.222 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-19T21:32:03.262 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:32:03.455 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T21:32:03.474 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-19T21:32:03.513 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:03.673 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-09-19T21:32:03.689 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-cephadm_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:03.692 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T21:32:03.709 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-19T21:32:03.750 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:32:03.756 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:03.866 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T21:32:03.885 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:03.924 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:04.063 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T21:32:04.081 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:04.101 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T21:32:04.118 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-19T21:32:04.154 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:04.159 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-19T21:32:04.236 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jwt. 2024-09-19T21:32:04.254 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-19T21:32:04.319 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:04.537 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T21:32:04.555 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-19T21:32:04.592 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T21:32:04.610 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:04.622 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-attr. 2024-09-19T21:32:04.628 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:04.641 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-19T21:32:04.677 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:04.680 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-19T21:32:04.982 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T21:32:04.997 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:05.032 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T21:32:05.041 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T21:32:05.048 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-19T21:32:05.058 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-19T21:32:05.064 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:05.110 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:32:05.137 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-19T21:32:05.409 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cryptography. 2024-09-19T21:32:05.427 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2024-09-19T21:32:05.429 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-19T21:32:05.443 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jwt. 2024-09-19T21:32:05.444 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-19T21:32:05.462 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-19T21:32:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:32:05.485 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:05.501 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:05.807 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-19T21:32:05.825 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-19T21:32:05.838 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-attr. 2024-09-19T21:32:05.855 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-19T21:32:05.864 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:05.887 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-19T21:32:05.903 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T21:32:05.920 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-19T21:32:05.961 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-19T21:32:06.192 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-packaging. 2024-09-19T21:32:06.209 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-19T21:32:06.250 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T21:32:06.250 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-19T21:32:06.267 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-19T21:32:06.317 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-19T21:32:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T21:32:06.372 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:06.414 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:06.586 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T21:32:06.603 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-19T21:32:06.644 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:06.653 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-09-19T21:32:06.671 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-19T21:32:06.728 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:06.947 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T21:32:06.951 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T21:32:06.965 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-19T21:32:06.967 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:07.005 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-19T21:32:07.009 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:07.089 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-19T21:32:07.105 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-19T21:32:07.147 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:07.375 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T21:32:07.391 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T21:32:07.433 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:07.467 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-packaging. 2024-09-19T21:32:07.484 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-19T21:32:07.534 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-19T21:32:07.686 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-underscore. 2024-09-19T21:32:07.703 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-19T21:32:07.769 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:32:07.887 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T21:32:07.906 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-19T21:32:07.944 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:07.980 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T21:32:08.000 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:08.047 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-19T21:32:08.059 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:08.064 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-19T21:32:08.113 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:32:08.281 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T21:32:08.298 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-19T21:32:08.372 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-19T21:32:08.441 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2024-09-19T21:32:08.458 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-19T21:32:08.499 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-19T21:32:08.751 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T21:32:08.770 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T21:32:08.809 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:08.843 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T21:32:08.860 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:08.902 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:09.087 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-underscore. 2024-09-19T21:32:09.104 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-19T21:32:09.144 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:32:09.447 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-19T21:32:09.464 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-19T21:32:09.505 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:32:09.858 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-09-19T21:32:09.875 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-19T21:32:09.916 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-19T21:32:10.287 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T21:32:10.299 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph. 2024-09-19T21:32:10.303 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:10.318 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../051-ceph_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:10.361 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:10.365 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-decorator. 2024-09-19T21:32:10.374 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:10.385 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-19T21:32:10.423 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:10.676 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-19T21:32:10.687 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T21:32:10.694 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-19T21:32:10.706 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:10.725 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:32:10.745 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:11.087 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-scipy. 2024-09-19T21:32:11.103 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-19T21:32:11.144 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-19T21:32:11.190 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T21:32:11.207 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:11.248 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:11.792 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-decorator. 2024-09-19T21:32:11.808 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-19T21:32:11.835 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephadm. 2024-09-19T21:32:11.849 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:11.853 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../054-cephadm_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:11.893 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:12.178 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-19T21:32:12.193 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-19T21:32:12.236 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:32:12.287 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T21:32:12.305 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-19T21:32:12.345 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-19T21:32:12.487 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-19T21:32:12.506 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-19T21:32:12.569 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:12.622 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-scipy. 2024-09-19T21:32:12.641 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-19T21:32:12.680 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-19T21:32:12.773 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T21:32:12.790 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-19T21:32:12.831 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:13.116 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T21:32:13.132 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-19T21:32:13.174 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-19T21:32:13.192 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T21:32:13.213 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:13.284 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:13.663 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jwt. 2024-09-19T21:32:13.679 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-19T21:32:13.745 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:14.077 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T21:32:14.081 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-19T21:32:14.090 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-attr. 2024-09-19T21:32:14.095 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-19T21:32:14.101 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-19T21:32:14.108 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-19T21:32:14.135 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:14.139 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:14.148 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-19T21:32:14.576 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T21:32:14.592 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-19T21:32:14.626 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-19T21:32:14.761 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T21:32:14.776 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-19T21:32:14.853 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-19T21:32:14.870 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T21:32:14.889 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:14.928 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:15.313 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-py. 2024-09-19T21:32:15.330 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-19T21:32:15.371 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:15.548 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T21:32:15.564 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-19T21:32:15.622 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:15.717 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-19T21:32:15.734 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-19T21:32:15.792 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:16.072 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T21:32:16.092 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-19T21:32:16.128 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-packaging. 2024-09-19T21:32:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-19T21:32:16.155 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:16.186 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-19T21:32:16.351 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T21:32:16.370 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:16.418 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:16.443 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T21:32:16.462 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-19T21:32:16.503 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:32:16.556 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T21:32:16.572 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-19T21:32:16.614 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:16.865 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T21:32:16.881 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-19T21:32:16.923 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:32:16.925 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T21:32:16.941 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-19T21:32:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-19T21:32:17.345 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T21:32:17.353 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T21:32:17.365 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-19T21:32:17.373 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T21:32:17.402 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-19T21:32:17.411 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:17.579 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T21:32:17.596 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-19T21:32:17.662 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:17.722 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-underscore. 2024-09-19T21:32:17.741 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-19T21:32:17.748 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T21:32:17.769 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-19T21:32:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:32:17.806 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-19T21:32:17.990 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T21:32:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-19T21:32:18.048 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:32:18.074 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-19T21:32:18.092 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-19T21:32:18.132 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:32:18.161 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-blinker. 2024-09-19T21:32:18.179 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-19T21:32:18.219 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:32:18.418 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T21:32:18.435 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-19T21:32:18.476 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:32:18.502 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-routes. 2024-09-19T21:32:18.519 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-19T21:32:18.560 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-19T21:32:18.564 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-19T21:32:18.583 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-19T21:32:18.622 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:32:18.908 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T21:32:18.924 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-19T21:32:18.955 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T21:32:18.971 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T21:32:18.973 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:18.987 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-19T21:32:19.007 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-19T21:32:19.013 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:19.029 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:32:19.345 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T21:32:19.363 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-19T21:32:19.366 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T21:32:19.383 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-19T21:32:19.403 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-19T21:32:19.424 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:32:19.761 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-blinker. 2024-09-19T21:32:19.777 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T21:32:19.777 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-19T21:32:19.793 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-19T21:32:19.819 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:32:19.834 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:32:20.156 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-19T21:32:20.172 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-19T21:32:20.214 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:32:20.425 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T21:32:20.442 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:20.483 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:20.485 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-decorator. 2024-09-19T21:32:20.501 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-19T21:32:20.542 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:20.610 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T21:32:20.626 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-19T21:32:20.661 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:32:20.844 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T21:32:20.861 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-19T21:32:20.902 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:32:20.912 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-19T21:32:20.932 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-19T21:32:20.970 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:32:21.006 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T21:32:21.024 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-19T21:32:21.064 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:32:21.289 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T21:32:21.298 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-scipy. 2024-09-19T21:32:21.307 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:21.315 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-19T21:32:21.347 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:21.389 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-19T21:32:21.393 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T21:32:21.411 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-19T21:32:21.450 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:32:21.642 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2024-09-19T21:32:21.658 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:21.700 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:22.002 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2024-09-19T21:32:22.020 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-19T21:32:22.023 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T21:32:22.040 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:22.060 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-19T21:32:22.080 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:22.397 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T21:32:22.414 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-19T21:32:22.417 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T21:32:22.435 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-19T21:32:22.455 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:32:22.475 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:32:22.824 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-19T21:32:22.826 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2024-09-19T21:32:22.843 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-ceph-test_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:22.844 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-19T21:32:22.870 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T21:32:22.874 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:22.884 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:22.888 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:22.945 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:23.248 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-09-19T21:32:23.264 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:23.306 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:23.469 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T21:32:23.486 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-19T21:32:23.527 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-19T21:32:23.576 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-09-19T21:32:23.592 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-19T21:32:23.633 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-19T21:32:23.945 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T21:32:23.961 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-19T21:32:23.972 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T21:32:23.993 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-19T21:32:24.003 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:32:24.030 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:24.360 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-09-19T21:32:24.376 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-ceph-test_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:24.418 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:24.831 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T21:32:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:24.922 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:26.109 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T21:32:26.125 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-19T21:32:26.201 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:26.515 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T21:32:26.533 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-19T21:32:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:32:26.951 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T21:32:26.970 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-19T21:32:27.009 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:32:27.363 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T21:32:27.380 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:27.455 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:27.481 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T21:32:27.498 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-19T21:32:27.539 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-19T21:32:27.736 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-19T21:32:27.752 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:27.794 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:27.869 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T21:32:27.889 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-19T21:32:27.935 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-19T21:32:28.273 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-blinker. 2024-09-19T21:32:28.289 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-19T21:32:28.314 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T21:32:28.330 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:32:28.334 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:28.372 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:28.665 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T21:32:28.684 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:28.704 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-19T21:32:28.720 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-19T21:32:28.756 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:28.762 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:32:28.792 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T21:32:28.809 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-19T21:32:28.869 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:29.054 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-19T21:32:29.074 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:29.084 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T21:32:29.102 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-19T21:32:29.122 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:29.157 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:32:29.208 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T21:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-19T21:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:32:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T21:32:29.519 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-19T21:32:29.560 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:32:29.644 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-19T21:32:29.662 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:29.694 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:29.709 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T21:32:29.727 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:29.767 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:29.949 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T21:32:29.966 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-19T21:32:30.006 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:32:30.196 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T21:32:30.233 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-19T21:32:30.271 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:30.602 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T21:32:30.618 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-19T21:32:30.662 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T21:32:30.679 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:30.687 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:32:30.719 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:31.072 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-19T21:32:31.088 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:31.106 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T21:32:31.124 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-19T21:32:31.130 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:31.172 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-19T21:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-19T21:32:31.502 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-19T21:32:31.542 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T21:32:31.561 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:31.625 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webencodings. 2024-09-19T21:32:31.864 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-19T21:32:31.886 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package jq. 2024-09-19T21:32:31.904 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-19T21:32:31.905 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:31.936 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:32.223 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package socat. 2024-09-19T21:32:32.237 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-html5lib. 2024-09-19T21:32:32.240 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-19T21:32:32.258 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-19T21:32:32.280 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-19T21:32:32.307 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-19T21:32:32.651 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T21:32:32.670 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-19T21:32:32.681 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-19T21:32:32.702 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-19T21:32:32.708 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:32:32.752 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:32:32.868 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-19T21:32:32.888 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-19T21:32:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-19T21:32:33.113 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-test. 2024-09-19T21:32:33.133 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../094-ceph-test_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:33.170 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:33.250 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T21:32:33.268 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T21:32:33.290 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webencodings. 2024-09-19T21:32:33.310 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-19T21:32:33.317 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T21:32:33.390 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:33.645 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T21:32:33.664 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-19T21:32:33.712 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:33.717 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-html5lib. 2024-09-19T21:32:33.734 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-19T21:32:33.776 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-19T21:32:34.073 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T21:32:34.090 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-19T21:32:34.123 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:34.145 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-19T21:32:34.162 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-19T21:32:34.203 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:32:34.484 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T21:32:34.501 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-19T21:32:34.542 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:34.715 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T21:32:34.735 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T21:32:34.781 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T21:32:34.828 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T21:32:34.845 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-19T21:32:34.911 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:35.118 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T21:32:35.138 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-19T21:32:35.176 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:35.540 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T21:32:35.559 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-19T21:32:35.614 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:35.626 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2024-09-19T21:32:35.644 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-radosgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:35.959 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T21:32:35.976 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-19T21:32:36.017 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:36.320 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T21:32:36.339 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-19T21:32:36.378 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:37.104 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T21:32:37.123 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-09-19T21:32:37.124 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:37.142 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-radosgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:37.162 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:37.181 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:37.394 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T21:32:37.413 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-855-g19e2a2be-1focal_all.deb ... 2024-09-19T21:32:37.431 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-nbd. 2024-09-19T21:32:37.449 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-rbd-nbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:37.452 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:37.514 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:37.825 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-19T21:32:37.907 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-19T21:32:37.926 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:37.956 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:32:38.264 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:32:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:38.543 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T21:32:38.560 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:38.592 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:38.601 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:38.619 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T21:32:38.636 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:38.677 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:38.980 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-nbd. 2024-09-19T21:32:38.997 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-rbd-nbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:39.038 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:39.057 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T21:32:39.075 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-19T21:32:39.125 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:39.134 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2024-09-19T21:32:39.323 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:39.385 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-19T21:32:39.449 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:32:39.487 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T21:32:39.504 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-19T21:32:39.545 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:32:39.710 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:32:39.784 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:39.869 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:32:39.992 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-19T21:32:39.995 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:40.011 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-19T21:32:40.050 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:40.110 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:40.221 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:40.236 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-19T21:32:40.346 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:40.664 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-19T21:32:40.822 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-09-19T21:32:40.986 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:41.083 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:41.137 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:32:41.393 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:41.496 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:41.669 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:32:41.696 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-19T21:32:41.714 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-19T21:32:41.770 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-19T21:32:41.789 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:41.890 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-19T21:32:41.953 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:42.016 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:42.132 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webencodings. 2024-09-19T21:32:42.151 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-19T21:32:42.190 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:42.263 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:42.351 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-19T21:32:42.544 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-html5lib. 2024-09-19T21:32:42.561 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-19T21:32:42.564 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:42.602 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-19T21:32:42.715 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:32:42.795 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-19T21:32:43.002 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-19T21:32:43.007 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:43.040 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:32:43.113 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:43.325 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:32:43.423 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:32:43.451 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:43.544 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T21:32:43.561 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T21:32:43.593 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T21:32:43.716 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:43.852 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-19T21:32:43.964 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T21:32:43.983 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-19T21:32:43.991 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:44.030 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:44.154 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:44.301 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:44.408 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T21:32:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-19T21:32:44.460 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:32:44.491 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:44.753 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:44.836 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T21:32:44.857 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-19T21:32:44.889 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:44.894 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:45.054 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:32:45.180 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:45.208 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T21:32:45.228 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-19T21:32:45.266 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:45.356 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:45.648 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-19T21:32:45.983 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-19T21:32:46.276 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-19T21:32:46.487 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package radosgw. 2024-09-19T21:32:46.499 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:46.506 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../110-radosgw_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:46.545 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:46.586 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:32:46.703 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:46.810 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:46.821 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:47.122 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:32:47.168 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:47.461 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:47.532 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:32:47.804 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-19T21:32:47.900 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:32:48.023 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T21:32:48.041 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:48.081 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-19T21:32:48.302 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-19T21:32:48.368 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package rbd-nbd. 2024-09-19T21:32:48.372 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:32:48.387 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../112-rbd-nbd_18.2.4-855-g19e2a2be-1focal_amd64.deb ... 2024-09-19T21:32:48.419 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:48.450 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:48.548 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:48.674 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:48.754 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:48.778 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-19T21:32:48.888 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-19T21:32:49.001 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:32:49.082 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-19T21:32:49.233 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-19T21:32:49.248 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:32:49.359 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-19T21:32:49.383 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:32:49.444 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:32:49.525 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-19T21:32:49.586 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:49.795 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:32:49.843 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:32:50.147 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-19T21:32:50.153 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:50.246 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user cephadm....done 2024-09-19T21:32:50.262 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:32:50.273 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:50.388 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:32:50.417 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:50.544 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-19T21:32:50.591 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:50.756 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:32:50.758 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-19T21:32:50.937 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T21:32:51.110 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:32:51.133 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-19T21:32:51.230 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-19T21:32:51.236 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:32:51.346 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-19T21:32:51.381 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-19T21:32:51.434 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:32:51.507 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-19T21:32:51.560 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:32:51.655 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:32:51.861 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-19T21:32:51.892 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-19T21:32:51.981 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:52.141 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:32:52.145 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:32:52.267 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:32:52.344 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-19T21:32:52.471 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:32:52.610 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:32:52.646 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-19T21:32:52.852 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-19T21:32:52.919 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-19T21:32:52.955 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-19T21:32:53.149 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:32:53.212 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:32:53.233 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-19T21:32:53.275 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:32:53.338 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:32:53.599 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-argparse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:53.618 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:32:53.641 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-19T21:32:53.909 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:53.974 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:32:53.994 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:32:54.035 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-19T21:32:54.104 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:32:54.230 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:32:54.329 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:32:54.353 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-19T21:32:54.606 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:32:54.645 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-19T21:32:54.693 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-19T21:32:54.804 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-19T21:32:54.915 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-19T21:32:55.011 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:32:55.136 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:32:55.206 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-19T21:32:55.454 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:32:55.566 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-19T21:32:55.822 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:32:55.858 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-19T21:32:55.982 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:32:56.108 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:32:56.226 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-19T21:32:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:32:56.429 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:56.589 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-19T21:32:56.611 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-19T21:32:56.685 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-19T21:32:56.970 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-19T21:32:56.974 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-19T21:32:57.283 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:32:57.316 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-19T21:32:57.614 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-19T21:32:57.618 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:57.744 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-19T21:32:57.773 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-19T21:32:57.891 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-19T21:32:58.111 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:32:58.141 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:32:58.259 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-19T21:32:58.300 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-19T21:32:58.488 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-19T21:32:58.668 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-19T21:32:58.677 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-19T21:32:58.873 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:32:58.952 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:32:59.087 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-19T21:32:59.208 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:32:59.262 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:32:59.388 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-19T21:32:59.497 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-19T21:32:59.623 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:32:59.731 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:32:59.983 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:33:00.057 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-19T21:33:00.118 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-19T21:33:00.288 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:33:00.442 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:33:00.447 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-19T21:33:00.503 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-19T21:33:00.582 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:33:00.629 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-19T21:33:00.708 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-19T21:33:00.777 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:33:00.779 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-19T21:33:01.042 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:33:01.081 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-19T21:33:01.168 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-19T21:33:01.449 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:33:01.486 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:33:01.575 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-19T21:33:01.734 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-19T21:33:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T21:33:01.840 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:33:01.966 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-19T21:33:02.069 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-19T21:33:02.117 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:33:02.122 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:33:02.227 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-19T21:33:02.420 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-19T21:33:02.536 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:33:02.541 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:02.662 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-19T21:33:02.692 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:33:02.730 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-19T21:33:02.818 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:02.889 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T21:33:02.944 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:03.005 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:33:03.189 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:03.190 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-19T21:33:03.315 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-19T21:33:03.339 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T21:33:03.491 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-19T21:33:03.633 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:03.691 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:33:03.759 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-19T21:33:03.876 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-19T21:33:04.101 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.102 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.212 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.219 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:33:04.237 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-19T21:33:04.345 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.371 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.504 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.553 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-19T21:33:04.602 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:04.602 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:04.688 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-19T21:33:04.749 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:04.875 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-19T21:33:05.056 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:05.072 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-19T21:33:05.193 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:05.196 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:05.302 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-19T21:33:05.317 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:33:05.457 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-19T21:33:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-19T21:33:05.649 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:05.735 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-19T21:33:05.780 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:05.906 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.111 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-19T21:33:06.149 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:06.149 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:06.296 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.422 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.583 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.587 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-19T21:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.716 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.734 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.850 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:06.859 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:33:07.009 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:07.579 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2024-09-19T21:33:07.805 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:07.939 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:07.972 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2024-09-19T21:33:08.074 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:08.140 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2024-09-19T21:33:08.162 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T21:33:08.233 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:08.273 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:33:08.326 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T21:33:08.359 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:08.402 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-19T21:33:08.485 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T21:33:08.820 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-19T21:33:09.013 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-09-19T21:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:09.130 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-19T21:33:09.229 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:09.390 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-09-19T21:33:09.477 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-19T21:33:09.583 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-09-19T21:33:09.606 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T21:33:09.607 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-19T21:33:09.732 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:09.732 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:09.764 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T21:33:10.033 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-19T21:33:10.071 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T21:33:10.164 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:10.369 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-19T21:33:10.529 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:10.587 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T21:33:10.629 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:10.803 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-19T21:33:11.024 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:11.065 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:11.066 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:11.179 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-19T21:33:11.243 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:11.243 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:11.523 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:11.709 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:11.945 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:11.945 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:12.035 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T21:33:12.268 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-19T21:33:12.394 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:12.436 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-19T21:33:12.500 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:12.562 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-19T21:33:12.713 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-19T21:33:12.724 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:12.725 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:12.917 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:12.917 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:13.031 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-19T21:33:13.169 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:13.182 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-19T21:33:13.371 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:13.395 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:13.395 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:13.475 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-19T21:33:13.496 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:13.641 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:13.777 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T21:33:13.853 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:13.893 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:13.893 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:14.128 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-19T21:33:14.330 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:14.330 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:14.342 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.477 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.505 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librados2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.644 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.656 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-19T21:33:14.755 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.782 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librgw2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.787 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.881 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:14.925 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libsqlite3-mod-ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:15.041 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:15.153 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs2 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:15.157 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-19T21:33:15.261 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:15.261 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:15.301 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-19T21:33:15.308 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-19T21:33:15.622 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libradosstriper1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:15.653 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-19T21:33:15.708 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:15.748 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-19T21:33:15.851 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.010 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.100 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librbd1 (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.153 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-modules-core (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.386 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:16.531 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-19T21:33:16.620 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:16.620 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T21:33:16.699 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-19T21:33:17.289 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs-dev (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:17.389 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rados (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:17.411 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-19T21:33:17.548 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-19T21:33:18.511 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:18.637 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up rbd-fuse (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:18.796 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up rbd-nbd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:18.798 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-19T21:33:18.906 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:19.057 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cephfs (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:19.183 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-common (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:19.454 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T21:33:19.460 DEBUG:teuthology.parallel:result is None 2024-09-19T21:33:19.828 INFO:teuthology.orchestra.run.smithi033.stdout:Adding group ceph....done 2024-09-19T21:33:20.238 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user ceph....done 2024-09-19T21:33:20.406 INFO:teuthology.orchestra.run.smithi033.stdout:Setting system user ceph properties....done 2024-09-19T21:33:20.415 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-19T21:33:20.430 INFO:teuthology.orchestra.run.smithi033.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T21:33:20.586 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T21:33:20.910 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T21:33:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T21:33:21.018 DEBUG:teuthology.parallel:result is None 2024-09-19T21:33:21.377 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-test (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:21.528 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up radosgw (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:22.039 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:22.040 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T21:33:22.489 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-base (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:22.906 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T21:33:23.374 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mds (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:23.614 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:23.615 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T21:33:24.101 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:24.337 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:24.338 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T21:33:24.819 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-osd (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:25.301 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:25.301 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T21:33:25.787 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-k8sevents (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:25.913 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:26.081 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mon (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:26.292 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T21:33:26.784 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-cephadm (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:26.925 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:27.085 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-dashboard (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:27.361 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-volume (18.2.4-855-g19e2a2be-1focal) ... 2024-09-19T21:33:27.723 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-19T21:33:27.908 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-19T21:33:28.209 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-19T21:33:31.104 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-19T21:33:31.744 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T21:33:31.749 DEBUG:teuthology.parallel:result is None 2024-09-19T21:33:31.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:33:31.886 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2024-09-19T21:33:31.910 INFO:teuthology.orchestra.run.smithi012.stdout:18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:31.911 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:31.911 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1focal is installed. 2024-09-19T21:33:31.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:33:32.048 DEBUG:teuthology.orchestra.run.smithi033:> dpkg-query -W -f '${Version}' ceph 2024-09-19T21:33:32.072 INFO:teuthology.orchestra.run.smithi033.stdout:18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:32.072 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:32.072 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1focal is installed. 2024-09-19T21:33:32.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-19T21:33:32.209 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-09-19T21:33:32.233 INFO:teuthology.orchestra.run.smithi122.stdout:18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:32.233 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855-g19e2a2be-1focal 2024-09-19T21:33:32.233 INFO:teuthology.task.install:The correct ceph version 18.2.4-855-g19e2a2be-1focal is installed. 2024-09-19T21:33:32.235 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T21:33:32.235 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:32.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T21:33:32.252 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:32.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T21:33:32.270 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:32.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T21:33:32.286 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T21:33:32.287 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:32.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T21:33:32.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T21:33:32.315 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:32.315 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T21:33:32.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T21:33:32.386 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:32.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T21:33:32.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T21:33:32.413 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T21:33:32.413 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:32.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T21:33:32.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T21:33:32.488 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:32.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T21:33:32.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T21:33:32.514 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:32.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T21:33:32.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T21:33:32.587 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T21:33:32.588 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:32.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T21:33:32.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T21:33:32.615 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:32.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T21:33:32.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T21:33:32.646 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:32.647 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T21:33:32.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T21:33:32.719 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T21:33:32.800 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T21:33:32.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2024-09-19T21:33:32.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-09-19T21:33:32.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-19T21:33:32.820 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T21:33:32.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T21:33:32.822 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T21:33:32.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T21:33:32.876 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T21:33:32.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T21:33:32.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T21:33:32.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T21:33:32.944 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T21:33:32.944 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T21:33:32.944 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850', 'branch': 'reef', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909850', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 4623, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'sleep_before_teardown': 0, 'subset': '80/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7', 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_22:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T21:33:32.945 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T21:33:32.947 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T21:33:32.949 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T21:33:32.954 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T21:33:32.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T21:33:32.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T21:33:33.013 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.013 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T21:33:33.020 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T21:33:33.020 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-09-19T21:33:33.027 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-19 21:29:42.196054263 +0000 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-19 21:29:42.192054355 +0000 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-19 21:29:42.192054355 +0000 2024-09-19T21:33:33.028 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-19T21:33:33.028 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T21:33:33.039 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-19T21:33:33.040 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-19T21:33:33.040 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000432707 s, 1.2 MB/s 2024-09-19T21:33:33.041 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T21:33:33.093 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-19 21:29:42.860038965 +0000 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-19 21:29:42.640044035 +0000 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-19 21:29:42.640044035 +0000 2024-09-19T21:33:33.100 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-19T21:33:33.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T21:33:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-19T21:33:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-19T21:33:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000424917 s, 1.2 MB/s 2024-09-19T21:33:33.113 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T21:33:33.165 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-09-19T21:33:33.172 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T21:33:33.172 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-19 21:29:43.284029197 +0000 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-19 21:29:43.068034174 +0000 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-19 21:29:43.068034174 +0000 2024-09-19T21:33:33.173 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-19T21:33:33.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T21:33:33.187 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-19T21:33:33.187 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-19T21:33:33.187 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000399481 s, 1.3 MB/s 2024-09-19T21:33:33.188 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T21:33:33.195 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-09-19T21:33:33.243 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T21:33:33.243 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.243 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-09-19T21:33:33.243 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.243 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-19 21:29:43.736018784 +0000 2024-09-19T21:33:33.244 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-19 21:29:43.516023853 +0000 2024-09-19T21:33:33.244 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-19 21:29:43.516023853 +0000 2024-09-19T21:33:33.244 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-19T21:33:33.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T21:33:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-19T21:33:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-19T21:33:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000368544 s, 1.4 MB/s 2024-09-19T21:33:33.257 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T21:33:33.265 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:33.265 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:33.265 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T21:33:33.272 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T21:33:33.272 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-09-19T21:33:33.279 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-19 21:29:42.606418282 +0000 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-19 21:29:42.402422987 +0000 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-19 21:29:42.402422987 +0000 2024-09-19T21:33:33.280 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-19T21:33:33.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T21:33:33.293 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-19T21:33:33.293 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-19T21:33:33.293 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000390406 s, 1.3 MB/s 2024-09-19T21:33:33.294 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T21:33:33.344 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-19 21:29:42.850412656 +0000 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-19 21:29:42.846412747 +0000 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-19 21:29:42.846412747 +0000 2024-09-19T21:33:33.351 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-19T21:33:33.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T21:33:33.366 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-19T21:33:33.367 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-19T21:33:33.367 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000376461 s, 1.4 MB/s 2024-09-19T21:33:33.368 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T21:33:33.420 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-09-19T21:33:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T21:33:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.427 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-19T21:33:33.427 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.428 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-19 21:29:43.270402970 +0000 2024-09-19T21:33:33.428 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-19 21:29:43.266403061 +0000 2024-09-19T21:33:33.428 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-19 21:29:43.266403061 +0000 2024-09-19T21:33:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-19T21:33:33.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T21:33:33.440 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-19T21:33:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-19T21:33:33.441 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000395678 s, 1.3 MB/s 2024-09-19T21:33:33.442 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T21:33:33.492 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-09-19T21:33:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T21:33:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T21:33:33.499 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-09-19T21:33:33.499 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T21:33:33.499 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-19 21:29:43.970386826 +0000 2024-09-19T21:33:33.500 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-19 21:29:43.730392361 +0000 2024-09-19T21:33:33.500 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-19 21:29:43.730392361 +0000 2024-09-19T21:33:33.500 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-09-19T21:33:33.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T21:33:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-19T21:33:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-19T21:33:33.512 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000356196 s, 1.4 MB/s 2024-09-19T21:33:33.513 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T21:33:33.564 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:33.564 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-19T21:33:33.564 INFO:tasks.ceph:Generating config... 2024-09-19T21:33:33.565 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-19T21:33:33.565 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-19T21:33:33.565 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-19T21:33:33.565 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T21:33:33.566 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T21:33:33.566 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T21:33:33.566 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T21:33:33.566 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-19T21:33:33.566 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-19T21:33:33.567 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T21:33:33.567 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T21:33:33.568 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T21:33:33.609 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T21:33:33.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T21:33:33.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T21:33:33.681 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.c', '[v2:172.21.15.12:3301,v1:172.21.15.12:6790]'), ('mon.b', '172.21.15.33')] 2024-09-19T21:33:33.681 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,[v2:172.21.15.12:3301,v1:172.21.15.12:6790],172.21.15.33', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-19T21:33:33.682 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.682 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T21:33:33.688 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --addv c '[v2:172.21.15.12:3301,v1:172.21.15.12:6790]' --add b 172.21.15.33 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:33.773 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:33.773 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:33:33.773 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2024-09-19T21:33:33.773 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2024-09-19T21:33:33.771015+0000 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-19T21:33:33.771015+0000 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 16 (pacific) 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.b 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] mon.c 2024-09-19T21:33:33.774 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-19T21:33:33.775 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T21:33:33.782 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5c98f4ba-835c-4398-91b1-e251a78b0f2b... 2024-09-19T21:33:33.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T21:33:33.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T21:33:33.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.818 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T21:33:33.819 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T21:33:33.820 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,[v2:172.21.15.12:3301,v1:172.21.15.12:6790],172.21.15.33" 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon client directed command retry = 5 2024-09-19T21:33:33.821 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on pool no app = False 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject socket failures = 5000 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2024-09-19T21:33:33.822 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2024-09-19T21:33:33.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: bluefs allocator = hybrid 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore allocator = hybrid 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 1/20 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 1/20 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-19T21:33:33.824 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 4/10 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: osd blocked scrub grace period = 3600 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: osd max markdown count = 1000 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-19T21:33:33.825 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2024-09-19T21:33:33.826 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.827 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2024-09-19T21:33:33.828 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2024-09-19T21:33:33.829 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-19T21:33:33.829 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-19T21:33:33.829 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T21:33:33.829 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.830 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-19T21:33:33.831 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.832 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.12,[v2:172.21.15.12:3301,v1:172.21.15.12:6790],172.21.15.33" 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on pool no app = False 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 5000 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.833 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-19T21:33:33.834 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.835 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: bluefs allocator = hybrid 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = hybrid 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 1/20 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 1/20 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 4/10 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-19T21:33:33.836 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: osd blocked scrub grace period = 3600 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: osd max markdown count = 1000 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.837 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-19T21:33:33.838 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.839 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T21:33:33.840 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T21:33:33.842 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-09-19T21:33:33.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.12,[v2:172.21.15.12:3301,v1:172.21.15.12:6790],172.21.15.33" 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon client directed command retry = 5 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on pool no app = False 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: ms inject socket failures = 5000 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:33:33.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-09-19T21:33:33.845 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.846 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: bluefs allocator = hybrid 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore allocator = hybrid 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 1/20 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 1/20 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 4/10 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-09-19T21:33:33.847 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: osd blocked scrub grace period = 3600 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: osd max markdown count = 1000 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.848 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T21:33:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T21:33:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T21:33:33.851 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-09-19T21:33:33.851 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-09-19T21:33:33.851 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-09-19T21:33:33.852 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T21:33:33.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T21:33:33.946 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T21:33:33.946 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.947 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T21:33:33.951 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.951 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T21:33:33.956 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:33:33.956 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T21:33:33.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T21:33:33.978 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:33.978 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:34.027 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:33:34.027 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:34.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T21:33:34.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T21:33:34.055 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:34.055 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:34.060 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-19T21:33:34.060 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:34.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T21:33:34.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T21:33:34.087 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:34.088 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:34.134 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T21:33:34.135 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T21:33:34.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T21:33:34.186 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T21:33:34.199 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T21:33:34.199 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T21:33:34.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T21:33:34.240 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T21:33:34.259 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T21:33:34.259 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-19T21:33:34.259 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T21:33:34.275 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:34.275 INFO:tasks.ceph:role: osd.0 2024-09-19T21:33:34.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:33:34.276 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T21:33:34.848 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:34.849 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:34.850 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:34.850 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:34.851 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-19T21:33:34.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T21:33:34.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T21:33:34.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:34.961 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:34.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T21:33:35.019 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:35.019 INFO:tasks.ceph:role: osd.1 2024-09-19T21:33:35.019 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:33:35.019 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T21:33:35.527 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:35.527 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:35.527 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:35.527 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:35.528 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:35.530 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-19T21:33:35.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-19T21:33:35.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T21:33:35.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:35.614 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:35.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T21:33:35.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:35.626 INFO:tasks.ceph:role: osd.2 2024-09-19T21:33:35.626 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:33:35.626 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:36.097 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:36.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-19T21:33:36.098 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-19T21:33:36.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T21:33:36.158 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:36.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:36.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:36.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.225+0000 7ff4c9ff3880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T21:33:36.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.225+0000 7ff4c9ff3880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T21:33:36.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.225+0000 7ff4c9ff3880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-19T21:33:36.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.225+0000 7ff4c9ff3880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-19T21:33:36.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.225+0000 7ff4c9ff3880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-19T21:33:36.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:36.229+0000 7ff4c9ff3880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T21:33:39.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T21:33:39.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.313+0000 7fc434a5c880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T21:33:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.313+0000 7fc434a5c880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T21:33:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.313+0000 7fc434a5c880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-19T21:33:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.313+0000 7fc434a5c880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-19T21:33:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.313+0000 7fc434a5c880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-19T21:33:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:39.317+0000 7fc434a5c880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T21:33:42.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T21:33:42.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T21:33:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T21:33:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-19T21:33:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-19T21:33:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-19T21:33:42.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:33:42.397+0000 7f28a84a0880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T21:33:45.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T21:33:45.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T21:33:45.423 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:45.423 INFO:tasks.ceph:role: osd.3 2024-09-19T21:33:45.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:33:45.423 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T21:33:45.942 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:45.943 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:45.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-19T21:33:45.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-19T21:33:46.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T21:33:46.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:46.042 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:46.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-19T21:33:46.052 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:46.053 INFO:tasks.ceph:role: osd.4 2024-09-19T21:33:46.053 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:33:46.053 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:46.590 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:46.591 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:46.591 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:46.592 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-19T21:33:46.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-19T21:33:46.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-19T21:33:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:46.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:46.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-19T21:33:46.672 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-19T21:33:46.672 INFO:tasks.ceph:role: osd.5 2024-09-19T21:33:46.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:33:46.672 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T21:33:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T21:33:47.206 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T21:33:47.206 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T21:33:47.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-09-19T21:33:47.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-19T21:33:47.262 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-19T21:33:47.275 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T21:33:47.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:33:47.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.415+0000 7f5c6dd8a880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T21:33:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.419+0000 7f5c6dd8a880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T21:33:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.419+0000 7f5c6dd8a880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-19T21:33:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.419+0000 7f5c6dd8a880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-19T21:33:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.419+0000 7f5c6dd8a880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-19T21:33:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:47.419+0000 7f5c6dd8a880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T21:33:50.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T21:33:50.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:50.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-19T21:33:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-19T21:33:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-19T21:33:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-19T21:33:50.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-19T21:33:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:50.487+0000 7f93ba0e7880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-19T21:33:53.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-19T21:33:53.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-19T21:33:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-19T21:33:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-19T21:33:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-19T21:33:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-19T21:33:53.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-19T21:33:53.607+0000 7f21d3bce880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-19T21:33:56.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-19T21:33:56.625 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T21:33:56.625 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:56.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T21:33:56.642 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:56.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T21:33:56.701 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:33:56.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T21:33:56.712 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:56.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T21:33:56.728 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:56.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T21:33:56.738 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:56.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-19T21:33:56.753 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:33:56.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-19T21:33:56.768 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-19T21:33:56.768 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T21:33:56.775 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T21:33:56.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T21:33:56.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T21:33:56.789 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCwmOxmiQuQDRAAdLrNGBnOvnHegJTiwycMkQ== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCzmOxmgVvvEhAAlvi+9Ot+AA6QuJ3mAWUA2g== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC2mOxmxhzNFxAAKhPIojH8p7HozobVk/5whQ== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCumOxmXGEZCxAAmIHk3w2uG++amY2HpLVlFA== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC7mOxmQLz0GBAAT72i6h5y1T7vv3R4n/KBXw== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC+mOxmYKYcHRAAaW3V1ww8CLo8b49Ics888A== 2024-09-19T21:33:56.790 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-09-19T21:33:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDBmOxmiMJQJBAAGP5WcjMwFLVdIN8br6yXdQ== 2024-09-19T21:33:56.791 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-09-19T21:33:56.791 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCumOxmzSlZDhAAj5SRZgjYZJuwKw/xuzVUMA== 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCwmOxmiQuQDRAAdLrNGBnOvnHegJTiwycMkQ== 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCzmOxmgVvvEhAAlvi+9Ot+AA6QuJ3mAWUA2g== 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2024-09-19T21:33:56.792 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC2mOxmxhzNFxAAKhPIojH8p7HozobVk/5whQ== 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCumOxmXGEZCxAAmIHk3w2uG++amY2HpLVlFA== 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC7mOxmQLz0GBAAT72i6h5y1T7vv3R4n/KBXw== 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC+mOxmYKYcHRAAaW3V1ww8CLo8b49Ics888A== 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDBmOxmiMJQJBAAGP5WcjMwFLVdIN8br6yXdQ== 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-09-19T21:33:56.793 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCumOxmzSlZDhAAj5SRZgjYZJuwKw/xuzVUMA== 2024-09-19T21:33:56.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.942 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:56.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T21:33:56.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T21:33:57.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T21:33:57.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T21:33:57.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T21:33:57.295 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T21:33:57.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T21:33:57.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T21:33:57.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T21:33:57.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-19T21:33:57.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T21:33:57.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-19T21:33:57.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-19T21:33:57.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T21:33:57.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-19T21:33:57.693 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:57.695 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T21:33:57.701 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T21:33:57.701 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T21:33:57.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T21:33:57.704 INFO:tasks.ceph.mon.a:Started 2024-09-19T21:33:57.704 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-19T21:33:57.704 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-19T21:33:57.706 INFO:tasks.ceph.mon.c:Started 2024-09-19T21:33:57.706 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-19T21:33:57.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-19T21:33:57.745 INFO:tasks.ceph.mon.b:Started 2024-09-19T21:33:57.745 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T21:33:57.745 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T21:33:57.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T21:33:57.748 INFO:tasks.ceph.mgr.x:Started 2024-09-19T21:33:57.748 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T21:33:57.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T21:34:03.196 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-09-19T21:34:03.208 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T21:34:03.208 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T21:34:03.208 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:34:03.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T21:34:03.224 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:34:03.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T21:34:03.238 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-19T21:34:03.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T21:34:03.252 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:34:03.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T21:34:03.269 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:34:03.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-19T21:34:03.278 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-19T21:34:03.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-19T21:34:03.334 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:03.330+0000 7f66244552c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T21:34:03.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb 0 2024-09-19T21:34:03.542 INFO:teuthology.orchestra.run.smithi033.stdout:0 2024-09-19T21:34:03.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3 1 2024-09-19T21:34:03.763 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-09-19T21:34:03.777 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 585d0e6c-0092-4cdb-ae1a-6260630475d0 2 2024-09-19T21:34:03.985 INFO:teuthology.orchestra.run.smithi033.stdout:2 2024-09-19T21:34:03.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 41e181a0-e2f9-491b-9b3d-80f5e108577c 3 2024-09-19T21:34:04.203 INFO:teuthology.orchestra.run.smithi033.stdout:3 2024-09-19T21:34:04.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new 0c81c5a4-40c7-43cc-8e53-553d71bebe2b 4 2024-09-19T21:34:04.351 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:04.346+0000 7f66244552c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T21:34:04.411 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:04.406+0000 7f66244552c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T21:34:04.428 INFO:teuthology.orchestra.run.smithi033.stdout:4 2024-09-19T21:34:04.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd new c161c258-bee9-41b5-8257-1ba140f79b9f 5 2024-09-19T21:34:04.583 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:04.578+0000 7f66244552c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T21:34:04.650 INFO:teuthology.orchestra.run.smithi033.stdout:5 2024-09-19T21:34:04.663 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T21:34:04.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T21:34:04.665 INFO:tasks.ceph.osd.0:Started 2024-09-19T21:34:04.666 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T21:34:04.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T21:34:04.668 INFO:tasks.ceph.osd.1:Started 2024-09-19T21:34:04.668 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T21:34:04.668 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T21:34:04.670 INFO:tasks.ceph.osd.2:Started 2024-09-19T21:34:04.671 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T21:34:04.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T21:34:04.673 INFO:tasks.ceph.osd.3:Started 2024-09-19T21:34:04.673 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-19T21:34:04.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-19T21:34:04.677 INFO:tasks.ceph.osd.4:Started 2024-09-19T21:34:04.678 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T21:34:04.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T21:34:04.679 INFO:tasks.ceph.osd.5:Started 2024-09-19T21:34:04.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:04.825 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:04.822+0000 7f66244552c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T21:34:04.877 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:04.874+0000 7f66244552c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T21:34:04.882 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:04.882 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:04.640393+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:04.895 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T21:34:04.895 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T21:34:05.058 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:05.054+0000 7f66244552c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T21:34:05.124 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:05.122+0000 7f66244552c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T21:34:05.191 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:05.186+0000 7f66244552c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T21:34:05.291 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:05.285+0000 7fb49ffd3880 -1 Falling back to public interface 2024-09-19T21:34:05.297 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:05.294+0000 7f958e050880 -1 Falling back to public interface 2024-09-19T21:34:05.298 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:05.294+0000 7f8441288880 -1 Falling back to public interface 2024-09-19T21:34:05.298 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:05.294+0000 7fb8abddc880 -1 Falling back to public interface 2024-09-19T21:34:05.299 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:05.293+0000 7fb877c0e880 -1 Falling back to public interface 2024-09-19T21:34:05.299 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:05.293+0000 7fe5e5415880 -1 Falling back to public interface 2024-09-19T21:34:05.499 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:05.590 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:05.586+0000 7f66244552c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T21:34:05.691 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-19T21:34:05.691 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":8,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:04.640393+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:05.707 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-19T21:34:05.713 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:05.710+0000 7f66244552c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T21:34:06.059 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:06.054+0000 7f66244552c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T21:34:06.253 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:06.250+0000 7f66244552c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T21:34:06.318 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:06.314+0000 7f66244552c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T21:34:06.480 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:06.478+0000 7f66244552c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T21:34:07.260 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.258+0000 7f66244552c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T21:34:07.334 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.330+0000 7f66244552c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T21:34:07.505 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.502+0000 7f66244552c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T21:34:07.571 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.566+0000 7f66244552c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T21:34:07.646 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.642+0000 7f66244552c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T21:34:07.949 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:07.946+0000 7f66244552c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T21:34:08.025 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:08.022+0000 7f66244552c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T21:34:08.148 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:08.142+0000 7f958e050880 -1 osd.3 0 log_to_monitors true 2024-09-19T21:34:08.340 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:08.333+0000 7fb49ffd3880 -1 osd.1 0 log_to_monitors true 2024-09-19T21:34:08.760 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:08.754+0000 7f66244552c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T21:34:08.845 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:08.837+0000 7fe5e5415880 -1 osd.2 0 log_to_monitors true 2024-09-19T21:34:09.097 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:09.089+0000 7fb877c0e880 -1 osd.0 0 log_to_monitors true 2024-09-19T21:34:09.151 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:09.146+0000 7f8441288880 -1 osd.4 0 log_to_monitors true 2024-09-19T21:34:09.260 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:09.258+0000 7fb8abddc880 -1 osd.5 0 log_to_monitors true 2024-09-19T21:34:09.804 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:09.798+0000 7f660afcc700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T21:34:09.804 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:34:09.802+0000 7f660afcc700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T21:34:10.820 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:10.818+0000 7f9589e8f700 -1 osd.3 0 waiting for initial osdmap 2024-09-19T21:34:10.829 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:10.826+0000 7f958193e700 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:11.813 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:11.810+0000 7f843da4e700 -1 osd.4 0 waiting for initial osdmap 2024-09-19T21:34:11.825 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:11.822+0000 7f8434b76700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:12.198 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:12.193+0000 7fe5e1254700 -1 osd.2 0 waiting for initial osdmap 2024-09-19T21:34:12.208 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:12.201+0000 7fe5d8d03700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:12.265 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:12.261+0000 7fb49be12700 -1 osd.1 0 waiting for initial osdmap 2024-09-19T21:34:12.277 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:12.273+0000 7fb873a4d700 -1 osd.0 0 waiting for initial osdmap 2024-09-19T21:34:12.281 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:12.273+0000 7fb4938c1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:12.285 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:12.281+0000 7fb86b4fc700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:12.313 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:12.613 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-19T21:34:12.613 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":12,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:11.211188+0000","last_up_change":"2024-09-19T21:34:11.211188+0000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:12.627 DEBUG:teuthology.misc:1 of 6 OSDs are up 2024-09-19T21:34:12.673 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:12.670+0000 7fb8a85a2700 -1 osd.5 0 waiting for initial osdmap 2024-09-19T21:34:12.685 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:12.682+0000 7fb89f6ca700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T21:34:19.232 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:19.536 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-19T21:34:19.537 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":17,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:16.920942+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:19.550 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-19T21:34:19.550 INFO:tasks.ceph:Creating RBD pool 2024-09-19T21:34:19.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-19T21:34:19.957 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'rbd' created 2024-09-19T21:34:19.968 DEBUG:teuthology.orchestra.run.smithi012:> rbd --cluster ceph pool init rbd 2024-09-19T21:34:22.967 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T21:34:22.968 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T21:34:22.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T21:34:22.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T21:34:22.968 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T21:34:23.311 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:23.324 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3719794114},{"type":"v1","addr":"172.21.15.33:6825","nonce":3719794114}]},"active_addr":"172.21.15.33:6825/3719794114","active_change":"2024-09-19T21:34:08.781283+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1848824541}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1360695049}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2384394218}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3903264709}]}]} 2024-09-19T21:34:23.326 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T21:34:23.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T21:34:23.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:23.631 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:22.946310+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T21:34:19.884271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:23.645 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T21:34:23.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:34:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:23.952 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:34:22.946310+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:34:04.640393+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T21:34:19.884271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:34:23.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T21:34:23.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T21:34:23.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T21:34:23.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T21:34:23.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T21:34:23.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T21:34:24.133 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:24.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T21:34:24.141 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:24.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T21:34:24.156 INFO:teuthology.orchestra.run.smithi012.stdout:51539607557 2024-09-19T21:34:24.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T21:34:24.168 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:24.168 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T21:34:24.180 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:24.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T21:34:24.193 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:24.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T21:34:24.493 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:24.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.0 2024-09-19T21:34:24.566 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:24.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.4 2024-09-19T21:34:24.582 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:24.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2024-09-19T21:34:24.623 INFO:teuthology.orchestra.run.smithi012.stdout:51539607556 2024-09-19T21:34:24.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.3 2024-09-19T21:34:24.669 INFO:teuthology.orchestra.run.smithi012.stdout:55834574850 2024-09-19T21:34:24.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.2 2024-09-19T21:34:24.700 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:24.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2024-09-19T21:34:25.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T21:34:25.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T21:34:25.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T21:34:25.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T21:34:25.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T21:34:25.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T21:34:25.921 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:25.924 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:25.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-19T21:34:25.936 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:25.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-09-19T21:34:25.938 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:26.018 INFO:teuthology.orchestra.run.smithi012.stdout:51539607557 2024-09-19T21:34:26.028 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:26.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.3 2024-09-19T21:34:26.032 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:26.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-09-19T21:34:26.041 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:26.085 INFO:teuthology.orchestra.run.smithi012.stdout:55834574852 2024-09-19T21:34:26.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-09-19T21:34:26.099 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:26.125 INFO:teuthology.orchestra.run.smithi012.stdout:55834574854 2024-09-19T21:34:26.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-09-19T21:34:26.138 DEBUG:teuthology.parallel:result is None 2024-09-19T21:34:26.138 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T21:34:26.139 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:26.426 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:26.436 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-19T21:34:24.798222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":172652,"kb_used_data":2308,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058388,"statfs":{"total":579820584960,"available":579643789312,"internally_reserved":0,"allocated":2363392,"data_stored":1572850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.852488"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.953808+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:22.953808+0000","last_peered":"2024-09-19T21:34:22.953808+0000","last_clean":"2024-09-19T21:34:22.953808+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:22.953808+0000","last_undegraded":"2024-09-19T21:34:22.953808+0000","last_fullsized":"2024-09-19T21:34:22.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.956831+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:22.956831+0000","last_peered":"2024-09-19T21:34:22.956831+0000","last_clean":"2024-09-19T21:34:22.956831+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:22.956831+0000","last_undegraded":"2024-09-19T21:34:22.956831+0000","last_fullsized":"2024-09-19T21:34:22.956831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954340+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:22.954340+0000","last_peered":"2024-09-19T21:34:22.954340+0000","last_clean":"2024-09-19T21:34:22.954340+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:22.954340+0000","last_undegraded":"2024-09-19T21:34:22.954340+0000","last_fullsized":"2024-09-19T21:34:22.954340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.706019+0000","last_change":"2024-09-19T21:34:23.706666+0000","last_active":"2024-09-19T21:34:23.706019+0000","last_peered":"2024-09-19T21:34:23.706019+0000","last_clean":"2024-09-19T21:34:23.706019+0000","last_became_active":"2024-09-19T21:34:20.959962+0000","last_became_peered":"2024-09-19T21:34:20.959962+0000","last_unstale":"2024-09-19T21:34:23.706019+0000","last_undegraded":"2024-09-19T21:34:23.706019+0000","last_fullsized":"2024-09-19T21:34:23.706019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:40.153065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845699,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.958180+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:22.958180+0000","last_peered":"2024-09-19T21:34:22.958180+0000","last_clean":"2024-09-19T21:34:22.958180+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:22.958180+0000","last_undegraded":"2024-09-19T21:34:22.958180+0000","last_fullsized":"2024-09-19T21:34:22.958180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.704675+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:23.704675+0000","last_peered":"2024-09-19T21:34:23.704675+0000","last_clean":"2024-09-19T21:34:23.704675+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:23.704675+0000","last_undegraded":"2024-09-19T21:34:23.704675+0000","last_fullsized":"2024-09-19T21:34:23.704675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.702361+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:23.702361+0000","last_peered":"2024-09-19T21:34:23.702361+0000","last_clean":"2024-09-19T21:34:23.702361+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:23.702361+0000","last_undegraded":"2024-09-19T21:34:23.702361+0000","last_fullsized":"2024-09-19T21:34:23.702361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954083+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:22.954083+0000","last_peered":"2024-09-19T21:34:22.954083+0000","last_clean":"2024-09-19T21:34:22.954083+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:22.954083+0000","last_undegraded":"2024-09-19T21:34:22.954083+0000","last_fullsized":"2024-09-19T21:34:22.954083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-19T21:34:21.950758+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:21.950758+0000","last_peered":"2024-09-19T21:34:21.950758+0000","last_clean":"2024-09-19T21:34:21.950758+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:21.950758+0000","last_undegraded":"2024-09-19T21:34:21.950758+0000","last_fullsized":"2024-09-19T21:34:21.950758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":66174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":184320,"data_stored":61206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:26.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:26.745 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:26.749 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:26.760 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-19T21:34:24.798222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":172652,"kb_used_data":2308,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058388,"statfs":{"total":579820584960,"available":579643789312,"internally_reserved":0,"allocated":2363392,"data_stored":1572850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.852488"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.953808+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:22.953808+0000","last_peered":"2024-09-19T21:34:22.953808+0000","last_clean":"2024-09-19T21:34:22.953808+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:22.953808+0000","last_undegraded":"2024-09-19T21:34:22.953808+0000","last_fullsized":"2024-09-19T21:34:22.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.956831+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:22.956831+0000","last_peered":"2024-09-19T21:34:22.956831+0000","last_clean":"2024-09-19T21:34:22.956831+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:22.956831+0000","last_undegraded":"2024-09-19T21:34:22.956831+0000","last_fullsized":"2024-09-19T21:34:22.956831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954340+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:22.954340+0000","last_peered":"2024-09-19T21:34:22.954340+0000","last_clean":"2024-09-19T21:34:22.954340+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:22.954340+0000","last_undegraded":"2024-09-19T21:34:22.954340+0000","last_fullsized":"2024-09-19T21:34:22.954340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.706019+0000","last_change":"2024-09-19T21:34:23.706666+0000","last_active":"2024-09-19T21:34:23.706019+0000","last_peered":"2024-09-19T21:34:23.706019+0000","last_clean":"2024-09-19T21:34:23.706019+0000","last_became_active":"2024-09-19T21:34:20.959962+0000","last_became_peered":"2024-09-19T21:34:20.959962+0000","last_unstale":"2024-09-19T21:34:23.706019+0000","last_undegraded":"2024-09-19T21:34:23.706019+0000","last_fullsized":"2024-09-19T21:34:23.706019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:40.153065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845699,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.958180+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:22.958180+0000","last_peered":"2024-09-19T21:34:22.958180+0000","last_clean":"2024-09-19T21:34:22.958180+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:22.958180+0000","last_undegraded":"2024-09-19T21:34:22.958180+0000","last_fullsized":"2024-09-19T21:34:22.958180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.704675+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:23.704675+0000","last_peered":"2024-09-19T21:34:23.704675+0000","last_clean":"2024-09-19T21:34:23.704675+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:23.704675+0000","last_undegraded":"2024-09-19T21:34:23.704675+0000","last_fullsized":"2024-09-19T21:34:23.704675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.702361+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:23.702361+0000","last_peered":"2024-09-19T21:34:23.702361+0000","last_clean":"2024-09-19T21:34:23.702361+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:23.702361+0000","last_undegraded":"2024-09-19T21:34:23.702361+0000","last_fullsized":"2024-09-19T21:34:23.702361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954083+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:22.954083+0000","last_peered":"2024-09-19T21:34:22.954083+0000","last_clean":"2024-09-19T21:34:22.954083+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:22.954083+0000","last_undegraded":"2024-09-19T21:34:22.954083+0000","last_fullsized":"2024-09-19T21:34:22.954083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-19T21:34:21.950758+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:21.950758+0000","last_peered":"2024-09-19T21:34:21.950758+0000","last_clean":"2024-09-19T21:34:21.950758+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:21.950758+0000","last_undegraded":"2024-09-19T21:34:21.950758+0000","last_fullsized":"2024-09-19T21:34:21.950758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":66174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":184320,"data_stored":61206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:26.761 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T21:34:26.761 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T21:34:26.761 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T21:34:26.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T21:34:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:27.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T21:34:27.104 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T21:34:27.104 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-19T21:34:27.115 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-19T21:34:27.115 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}} 2024-09-19T21:34:27.115 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-19T21:34:27.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-19T21:34:27.436 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-19T21:34:27.437 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T21:34:27.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:27.742 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:27.744 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:27.755 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-19T21:34:26.798829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":172652,"kb_used_data":2308,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058388,"statfs":{"total":579820584960,"available":579643789312,"internally_reserved":0,"allocated":2363392,"data_stored":1572850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001697"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.953808+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:22.953808+0000","last_peered":"2024-09-19T21:34:22.953808+0000","last_clean":"2024-09-19T21:34:22.953808+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:22.953808+0000","last_undegraded":"2024-09-19T21:34:22.953808+0000","last_fullsized":"2024-09-19T21:34:22.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.956831+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:22.956831+0000","last_peered":"2024-09-19T21:34:22.956831+0000","last_clean":"2024-09-19T21:34:22.956831+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:22.956831+0000","last_undegraded":"2024-09-19T21:34:22.956831+0000","last_fullsized":"2024-09-19T21:34:22.956831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954340+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:22.954340+0000","last_peered":"2024-09-19T21:34:22.954340+0000","last_clean":"2024-09-19T21:34:22.954340+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:22.954340+0000","last_undegraded":"2024-09-19T21:34:22.954340+0000","last_fullsized":"2024-09-19T21:34:22.954340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.706019+0000","last_change":"2024-09-19T21:34:23.706666+0000","last_active":"2024-09-19T21:34:23.706019+0000","last_peered":"2024-09-19T21:34:23.706019+0000","last_clean":"2024-09-19T21:34:23.706019+0000","last_became_active":"2024-09-19T21:34:20.959962+0000","last_became_peered":"2024-09-19T21:34:20.959962+0000","last_unstale":"2024-09-19T21:34:23.706019+0000","last_undegraded":"2024-09-19T21:34:23.706019+0000","last_fullsized":"2024-09-19T21:34:23.706019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:40.153065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845699,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.958180+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:22.958180+0000","last_peered":"2024-09-19T21:34:22.958180+0000","last_clean":"2024-09-19T21:34:22.958180+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:22.958180+0000","last_undegraded":"2024-09-19T21:34:22.958180+0000","last_fullsized":"2024-09-19T21:34:22.958180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.704675+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:23.704675+0000","last_peered":"2024-09-19T21:34:23.704675+0000","last_clean":"2024-09-19T21:34:23.704675+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:23.704675+0000","last_undegraded":"2024-09-19T21:34:23.704675+0000","last_fullsized":"2024-09-19T21:34:23.704675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.702361+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:23.702361+0000","last_peered":"2024-09-19T21:34:23.702361+0000","last_clean":"2024-09-19T21:34:23.702361+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:23.702361+0000","last_undegraded":"2024-09-19T21:34:23.702361+0000","last_fullsized":"2024-09-19T21:34:23.702361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954083+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:22.954083+0000","last_peered":"2024-09-19T21:34:22.954083+0000","last_clean":"2024-09-19T21:34:22.954083+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:22.954083+0000","last_undegraded":"2024-09-19T21:34:22.954083+0000","last_fullsized":"2024-09-19T21:34:22.954083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-19T21:34:21.950758+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:21.950758+0000","last_peered":"2024-09-19T21:34:21.950758+0000","last_clean":"2024-09-19T21:34:21.950758+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:21.950758+0000","last_undegraded":"2024-09-19T21:34:21.950758+0000","last_fullsized":"2024-09-19T21:34:21.950758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":66174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31304,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340536,"statfs":{"total":96636764160,"available":96604708864,"internally_reserved":0,"allocated":184320,"data_stored":61206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:27.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:28.059 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:28.062 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:28.073 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-19T21:34:27.808060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":172656,"kb_used_data":2320,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":566058384,"statfs":{"total":579820584960,"available":579643785216,"internally_reserved":0,"allocated":2375680,"data_stored":1577818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.859735"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.953808+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:22.953808+0000","last_peered":"2024-09-19T21:34:22.953808+0000","last_clean":"2024-09-19T21:34:22.953808+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:22.953808+0000","last_undegraded":"2024-09-19T21:34:22.953808+0000","last_fullsized":"2024-09-19T21:34:22.953808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.956831+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:22.956831+0000","last_peered":"2024-09-19T21:34:22.956831+0000","last_clean":"2024-09-19T21:34:22.956831+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:22.956831+0000","last_undegraded":"2024-09-19T21:34:22.956831+0000","last_fullsized":"2024-09-19T21:34:22.956831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954340+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:22.954340+0000","last_peered":"2024-09-19T21:34:22.954340+0000","last_clean":"2024-09-19T21:34:22.954340+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:22.954340+0000","last_undegraded":"2024-09-19T21:34:22.954340+0000","last_fullsized":"2024-09-19T21:34:22.954340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.706019+0000","last_change":"2024-09-19T21:34:23.706666+0000","last_active":"2024-09-19T21:34:23.706019+0000","last_peered":"2024-09-19T21:34:23.706019+0000","last_clean":"2024-09-19T21:34:23.706019+0000","last_became_active":"2024-09-19T21:34:20.959962+0000","last_became_peered":"2024-09-19T21:34:20.959962+0000","last_unstale":"2024-09-19T21:34:23.706019+0000","last_undegraded":"2024-09-19T21:34:23.706019+0000","last_fullsized":"2024-09-19T21:34:23.706019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:40.153065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845699,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.958180+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:22.958180+0000","last_peered":"2024-09-19T21:34:22.958180+0000","last_clean":"2024-09-19T21:34:22.958180+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:22.958180+0000","last_undegraded":"2024-09-19T21:34:22.958180+0000","last_fullsized":"2024-09-19T21:34:22.958180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.704675+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:23.704675+0000","last_peered":"2024-09-19T21:34:23.704675+0000","last_clean":"2024-09-19T21:34:23.704675+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:23.704675+0000","last_undegraded":"2024-09-19T21:34:23.704675+0000","last_fullsized":"2024-09-19T21:34:23.704675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:23.702361+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:23.702361+0000","last_peered":"2024-09-19T21:34:23.702361+0000","last_clean":"2024-09-19T21:34:23.702361+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:23.702361+0000","last_undegraded":"2024-09-19T21:34:23.702361+0000","last_fullsized":"2024-09-19T21:34:23.702361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T21:34:22.954083+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:22.954083+0000","last_peered":"2024-09-19T21:34:22.954083+0000","last_clean":"2024-09-19T21:34:22.954083+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:22.954083+0000","last_undegraded":"2024-09-19T21:34:22.954083+0000","last_fullsized":"2024-09-19T21:34:22.954083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-19T21:34:21.950758+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:21.950758+0000","last_peered":"2024-09-19T21:34:21.950758+0000","last_clean":"2024-09-19T21:34:21.950758+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:21.950758+0000","last_undegraded":"2024-09-19T21:34:21.950758+0000","last_fullsized":"2024-09-19T21:34:21.950758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":66174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":656542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":200704,"data_stored":66193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":3,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31308,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":196608,"data_stored":66174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:28.074 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T21:34:28.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-19T21:34:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 22 2024-09-19T21:34:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 5c98f4ba-835c-4398-91b1-e251a78b0f2b 2024-09-19T21:34:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-19T21:34:03.067364+0000 2024-09-19T21:34:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-09-19T21:34:27.801359+0000 2024-09-19T21:34:28.381 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 6 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client jewel 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release reef 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 6 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6808/226723117,v1:172.21.15.12:6810/226723117] [v2:172.21.15.12:6812/226723117,v1:172.21.15.12:6814/226723117] exists,up 66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb 2024-09-19T21:34:28.382 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6800/1690053836,v1:172.21.15.12:6801/1690053836] [v2:172.21.15.12:6802/1690053836,v1:172.21.15.12:6803/1690053836] exists,up 4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3 2024-09-19T21:34:28.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6809/3902303328,v1:172.21.15.12:6811/3902303328] [v2:172.21.15.12:6813/3902303328,v1:172.21.15.12:6815/3902303328] exists,up 585d0e6c-0092-4cdb-ae1a-6260630475d0 2024-09-19T21:34:28.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 12 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/2014863703,v1:172.21.15.33:6801/2014863703] [v2:172.21.15.33:6802/2014863703,v1:172.21.15.33:6803/2014863703] exists,up 41e181a0-e2f9-491b-9b3d-80f5e108577c 2024-09-19T21:34:28.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6806/2893455664,v1:172.21.15.33:6810/2893455664] [v2:172.21.15.33:6813/2893455664,v1:172.21.15.33:6815/2893455664] exists,up 0c81c5a4-40c7-43cc-8e53-553d71bebe2b 2024-09-19T21:34:28.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6805/4275333014,v1:172.21.15.33:6808/4275333014] [v2:172.21.15.33:6812/4275333014,v1:172.21.15.33:6814/4275333014] exists,up c161c258-bee9-41b5-8257-1ba140f79b9f 2024-09-19T21:34:28.395 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6808/226723117,v1:172.21.15.12:6810/226723117] [v2:172.21.15.12:6812/226723117,v1:172.21.15.12:6814/226723117] exists,up 66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6800/1690053836,v1:172.21.15.12:6801/1690053836] [v2:172.21.15.12:6802/1690053836,v1:172.21.15.12:6803/1690053836] exists,up 4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6809/3902303328,v1:172.21.15.12:6811/3902303328] [v2:172.21.15.12:6813/3902303328,v1:172.21.15.12:6815/3902303328] exists,up 585d0e6c-0092-4cdb-ae1a-6260630475d0', 'osd.3 up in weight 1 up_from 12 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6800/2014863703,v1:172.21.15.33:6801/2014863703] [v2:172.21.15.33:6802/2014863703,v1:172.21.15.33:6803/2014863703] exists,up 41e181a0-e2f9-491b-9b3d-80f5e108577c', 'osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6806/2893455664,v1:172.21.15.33:6810/2893455664] [v2:172.21.15.33:6813/2893455664,v1:172.21.15.33:6815/2893455664] exists,up 0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6805/4275333014,v1:172.21.15.33:6808/4275333014] [v2:172.21.15.33:6812/4275333014,v1:172.21.15.33:6814/4275333014] exists,up c161c258-bee9-41b5-8257-1ba140f79b9f'] 2024-09-19T21:34:28.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mon.a", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster": "ceph", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "admin_socket_mode": "", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_client_required": "cephx, none", 2024-09-19T21:34:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_cluster_required": "cephx", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_debug": "true", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_service_required": "cephx", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_supported": "cephx", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio": "true", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_poll_ms": "250", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_reap_max": "16", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_async_discard": "false", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_block_size": "4096", 2024-09-19T21:34:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_inflight_ios": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_enable_discard": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_flock_retry": "3", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_inject_crash": "0", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring_hipri": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-19T21:34:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_type": "", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_alloc_size": "1048576", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_allocator": "hybrid", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_buffered_io": "true", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_for_zeros": "false", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_compact_log_sync": "false", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-19T21:34:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_max_log_runway": "4194304", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_max_prefetch": "1048576", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_min_flush_size": "524288", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_min_log_runway": "1048576", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_replay_recovery": "false", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_sync_write": "false", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-19T21:34:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_allocation_from_file": "true", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_allocator": "hybrid", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_create": "true", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_create": "false", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_path": "", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_size": "0", 2024-09-19T21:34:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_path": "", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_preallocate_file": "false", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_size": "107374182400", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_create": "false", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_path": "", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_size": "100663296", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs": "true", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T21:34:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_autotune": "true", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size": "0", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-19T21:34:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_type": "2q", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_clone_cow": "true", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_mode": "none", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-19T21:34:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_csum_type": "crc32c", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_freelist": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-19T21:34:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_small_allocations": "0", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_default_buffered_read": "true", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_default_buffered_write": "false", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-19T21:34:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fail_eio": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mount": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-19T21:34:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_umount": "false", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_ignore_data_csum": "false", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-19T21:34:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_op_age": "5.000000", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_alloc_size": "0", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size": "0", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_deferred_txc": "32", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size": "0", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_nid_prealloc": "1024", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-19T21:34:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_retry_disk_reads": "3", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_cf": "true", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_mem": "512", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-19T21:34:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_tracing": "false", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-19T21:34:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_zero_block_detection": "false", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-19T21:34:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_cluster_require_signatures": "false", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_cluster_require_version": "2", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_require_signatures": "false", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_require_version": "2", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_service_require_signatures": "false", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_service_require_version": "2", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_sign_messages": "true", 2024-09-19T21:34:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "chdir": "", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_acl_type": "", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_asio_thread_count": "2", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_cache_mid": "0.750000", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_cache_size": "16384", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_caps_release_delay": "5", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_check_pool_perm": "true", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_force_sync_read": "false", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_getattr_caps": "false", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_inject_features": "", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_inject_tick_delay": "0", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_die_on_failed_remount": "false", 2024-09-19T21:34:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_dirsize_rbytes": "true", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_force_lazyio": "false", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_fs": "", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_inject_release_failure": "false", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_max_inline_size": "4096", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mds_namespace": "", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_metadata": "", 2024-09-19T21:34:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_gid": "-1", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_timeout": "300", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_uid": "-1", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mountpoint": "/", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_notify_timeout": "10", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc": "true", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_dirty": "104857600", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_objects": "1000", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_size": "209715200", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_target_dirty": "8388608", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_permissions": "true", 2024-09-19T21:34:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_quota": "true", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_quota_df": "true", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_max_bytes": "0", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_max_periods": "4", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_min": "131072", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_reconnect_stale": "false", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_shutdown_timeout": "30", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_snapdir": ".snap", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_tick_interval": "1", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_trace": "", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_try_dentry_invalidate": "false", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_use_faked_inos": "false", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_use_random_mds": "false", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog": "false", 2024-09-19T21:34:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog_port": "12201", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_monitors": "default=true", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog": "false", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog_level": "info", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "-", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_network": "", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_network_interface": "", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_isal": "false", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_level": "5", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_winsize": "-15", 2024-09-19T21:34:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zstd_level": "1", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_alien_op_num_threads": "6", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_seastar_smp": "1", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location": "", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location_hook": "", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location_hook_timeout": "10", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "daemonize": "false", 2024-09-19T21:34:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "daos_pool": "tank", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_alienstore": "0/5", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_allow_any_pool_priority": "false", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asok": "1/5", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asok_assert_abort": "false", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asserts_on_shutdown": "true", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_auth": "20/20", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bdev": "1/3", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bluefs": "1/5", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bluestore": "1/5", 2024-09-19T21:34:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_buffer": "0/1", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_ceph_exporter": "1/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cephfs_mirror": "0/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cephsqlite": "0/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_client": "0/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_compressor": "1/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_context": "0/1", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_crush": "1/1", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_crypto": "1/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cyanstore": "0/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_deliberately_leak_memory": "false", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_disable_randomized_ping": "false", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_dpdk": "1/5", 2024-09-19T21:34:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_eventtrace": "1/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_filer": "0/1", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_filestore": "1/3", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_finisher": "1/1", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_fuse": "1/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_heartbeat_testing_span": "0", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_heartbeatmap": "1/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_javaclient": "1/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_journal": "1/3", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_journaler": "0/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_kstore": "1/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_leveldb": "4/5", 2024-09-19T21:34:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_lockdep": "0/1", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mclock": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_balancer": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_locker": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_log": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_log_expire": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_migrator": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_memstore": "1/5", 2024-09-19T21:34:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mgr": "2/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mgrc": "1/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mon": "20/20", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_monc": "0/10", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_ms": "1/1", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_none": "0/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objclass": "0/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objectcacher": "0/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objecter": "0/1", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_optracker": "0/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_osd": "1/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_paxos": "20/20", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_perfcounter": "1/5", 2024-09-19T21:34:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_prioritycache": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rados": "0/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd": "0/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_mirror": "0/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_pwl": "0/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_replay": "0/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_refs": "0/0", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_reserver": "1/1", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_access": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_datacache": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_dbstore": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_flight": "1/5", 2024-09-19T21:34:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_sync": "1/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rocksdb": "4/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_backref": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_cache": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_cleaner": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_device": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_epm": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_journal": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_lba": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_odata": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_omap": "0/5", 2024-09-19T21:34:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_onode": "0/5", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_t": "0/5", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_tm": "0/5", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_striper": "0/1", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_test": "0/5", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_throttle": "1/1", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_timer": "0/1", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_tp": "0/5", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "device_failure_prediction_mode": "none", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-19T21:34:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_graylog": "false", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_journald": "false", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_stderr": "true", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_syslog": "false", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event_tracing": "false", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_addr": "0.0.0.0", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_http_port": "9926", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_prio_limit": "5", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_sort_metrics": "true", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_stats_period": "5", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "fake_statfs_for_testing": "0", 2024-09-19T21:34:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "fatal_signal_handlers": "true", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filer_max_purge_ops": "10", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filer_max_truncate_ops": "128", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_apply_finisher_threads": "1", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_blackhole": "false", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_btrfs_clone_range": "true", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_btrfs_snap": "true", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_caller_concurrency": "10", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_collect_device_partition_information": "true", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_commit_timeout": "600.000000", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_inject_read_err": "false", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_omap_check": "false", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-19T21:34:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_verify_split": "false", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_dump_file": "", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fadvise": "true", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fail_eio": "true", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fd_cache_shards": "16", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fd_cache_size": "128", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fiemap": "false", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fiemap_threshold": "4096", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_inject_stall": "0", 2024-09-19T21:34:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_parallel": "false", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_trailing": "false", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_writeahead": "false", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_kill_at": "0", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs": "0", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-19T21:34:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size": "0", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_merge_threshold": "-10", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_odsync_write": "false", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_backend": "rocksdb", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_backend_path": "", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-19T21:34:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_thread_timeout": "60", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_threads": "2", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_punch_hole": "false", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-19T21:34:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_ops": "50", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_seek_data_hole": "false", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_sloppy_crc": "false", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_splice": "false", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_split_multiple": "2", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_split_rand_factor": "20", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_update_to": "1000", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-19T21:34:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_enable": "true", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_xfs_extsize": "false", 2024-09-19T21:34:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_zfs_snap": "false", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fio_dir": "/tmp/fio", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_allow_other": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_atomic_o_trunc": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_big_writes": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_debug": "false", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_default_permissions": "false", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_disable_pagecache": "false", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_max_write": "0", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_multithreaded": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_require_active_mds": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_set_user_groups": "true", 2024-09-19T21:34:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_move": "true", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_read": "true", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_write": "true", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_use_invalidate_cb": "true", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "gss_target_name": "ceph", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_file": "", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_inject_failure": "0", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_interval": "5", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "host": "smithi012", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-19T21:34:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "inject_early_sigterm": "false", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "jaeger_agent_port": "6799", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "jaeger_tracing_enable": "false", 2024-09-19T21:34:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_aio": "true", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_align_min_size": "65536", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_block_align": "true", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_block_size": "4096", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_dio": "true", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_discard": "false", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_force_aio": "false", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_ignore_corruption": "false", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_max_write_bytes": "10485760", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_max_write_entries": "100", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_replay_from": "0", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-19T21:34:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_write_header_frequency": "0", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_zero_on_create": "false", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_prefetch_periods": "10", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_prezero_periods": "5", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_write_head_interval": "15", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "keyfile": "", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_backend": "rocksdb", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_default_stripe_size": "65536", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_fsck_on_mount": "false", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_max_bytes": "67108864", 2024-09-19T21:34:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_max_ops": "512", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_nid_prealloc": "1024", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_onode_map_size": "1024", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_sync_submit_transaction": "false", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_sync_transaction": "false", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_block_size": "65536", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_bloom_size": "0", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_cache_size": "536870912", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_compact_on_mount": "false", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_compression": "false", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_log": "", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-19T21:34:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_max_open_files": "0", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_paranoid": "false", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "librados_thread_count": "2", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "lockdep": "false", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "lockdep_force_backtrace": "false", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_coarse_timestamps": "true", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_flush_on_exit": "false", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_graylog_host": "127.0.0.1", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_graylog_port": "12201", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_max_new": "1000", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_max_recent": "10000", 2024-09-19T21:34:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_stderr_prefix": "", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_stop_at_utilization": "0.970000", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": "true", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_graylog": "false", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_journald": "false", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_stderr": "false", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_syslog": "false", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "max_rotating_auth_attempts": "10", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_action_on_write_error": "1", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_alternate_name_max": "8192", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_asio_thread_count": "2", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_export_pin": "true", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_dirs": "true", 2024-09-19T21:34:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_interval": "5", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_interval": "10", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_max": "-1", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_max_until": "-1", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_merge_size": "50", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_midchunk": "0.300000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_min_start": "0.200000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_minchunk": "0.001000", 2024-09-19T21:34:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_mode": "0", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_need_max": "1.200000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_need_min": "0.800000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_bits": "3", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_size": "10000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_target_decay": "10.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_grace": "15.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_interval": "4.000000", 2024-09-19T21:34:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_mid": "0.700000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_release_free_interval": "10", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_reservation": "0.050000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_interval": "1", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_threshold": "262144", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_prealloc_inos": "1000", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-19T21:34:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_connect_bootstrapping": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_damage_table_max_entries": "10000", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_auth_pins": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_frag": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_scatterstat": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_subtrees": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_decay_halflife": "5.000000", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_default_dir_hash": "2", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_defer_session_stale": "true", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_deny_all_reconnect": "false", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_keys_per_op": "16384", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_max_commit_size": "10", 2024-09-19T21:34:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_max_entries": "0", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_prefetch": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_on_map": "false", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_early_reply": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_enable_op_tracker": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_enforce_unique_name": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_random": "true", 2024-09-19T21:34:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_health_summarize_threshold": "10", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_health_dummy": "false", 2024-09-19T21:34:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_migrator_session_race": "false", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_join_fs": "", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_journal_format": "1", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_create_at": "0", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_export_at": "0", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_import_at": "0", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_at": "0", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_expire_at": "0", 2024-09-19T21:34:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_replay_at": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_link_at": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_mdstable_at": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_openc_at": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_rename_at": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_events_per_segment": "1024", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_max_events": "-1", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_max_segments": "128", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_pause": "false", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_segment_size": "0", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_warn_factor": "2.000000", 2024-09-19T21:34:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_caps_per_client": "1048576", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_completed_flushes": "100000", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_completed_requests": "100000", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_export_size": "20971520", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_file_recover": "32", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_files": "64", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_ops": "8192", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_snaps_per_dir": "100", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_metrics_update_interval": "2", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_min_caps_per_client": "100", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_min_caps_working_set": "10000", 2024-09-19T21:34:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_numa_node": "-1", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_complaint_time": "30.000000", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_duration": "600", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_size": "20", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_slow_op_size": "20", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_log_threshold": "5", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_ping_grace": "15", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_ping_interval": "5", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-19T21:34:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_caps": "30000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_warning_threshold": "262144", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_replay_interval": "1.000000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_root_ino_gid": "0", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_root_ino_uid": "0", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-19T21:34:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_shutdown_check": "0", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_skip_ino": "0", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_max_uid": "4294967294", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_min_uid": "0", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_rstat": "false", 2024-09-19T21:34:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_standby_replay_damaged": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_symlink_recovery": "true", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_thrash_exports": "0", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_thrash_fragments": "0", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_tick_interval": "5.000000", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_valgrind_exit": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_verify_backtrace": "1", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_verify_scatter": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_wipe_sessions": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool_debug": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_device_bytes": "1073741824", 2024-09-19T21:34:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_page_set": "false", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_page_size": "65536", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_bytes": "134217728", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_messages": "512", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_disabled_modules": "", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_max_pg_num_change": "128", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mds_bytes": "134217728", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mds_messages": "128", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-19T21:34:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mon_bytes": "134217728", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mon_messages": "128", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_osd_bytes": "536870912", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_osd_messages": "8192", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_pool": "true", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_standby_modules": "true", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_stats_period": "5", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_stats_threshold": "5", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_tick_period": "2", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_allow_pool_delete": "true", 2024-09-19T21:34:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_allow_pool_size_one": "true", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_auth_validate_all_caps": "true", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_bytes": "104857600", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_directed_command_retry": "5", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_parallel": "3", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_log_interval": "1.000000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-19T21:34:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_ping_interval": "10.000000", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_file_level": "debug", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_file": "true", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_journald": "false", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-19T21:34:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_bootstrap": "false", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_start": "false", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_trim": "true", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cpu_threads": "4", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_crush_min_required_version": "hammer", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_daemon_bytes": "419430400", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-19T21:34:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_avail_crit": "5", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_avail_warn": "5", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_size_warn": "16106127360", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_json": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_transactions": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_extra_checks": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_quincy": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_reef": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-19T21:34:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_down_mkfs_grace": "120", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_election_default_strategy": "1", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_election_timeout": "5.000000", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ping_divisor": "2", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_enable_op_tracker": "true", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_fake_pool_delete": "false", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_force_quorum_join": "false", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_globalid_prealloc": "10000", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_detail_to_clog": "false", 2024-09-19T21:34:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_log_update_period": "5", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_max_detail": "50", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog": "true", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog_interval": "600", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_host": "172.21.15.12,[v2:172.21.15.12:3301,v1:172.21.15.12:6790],172.21.15.33", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_host_override": "", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_initial_members": "", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-19T21:34:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease": "5.000000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_full_interval": "50", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_max": "10000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_max_summary": "50", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_log_epochs": "500", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_osd": "10000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_pg_per_osd": "10000", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_pool_pg_num": "65536", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-19T21:34:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_force_trim_to": "0", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_skip_sanity": "false", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_memory_autotune": "true", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_memory_target": "2147483648", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_beacon_grace": "30", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_digest_period": "5", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_inactive_grace": "60", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_min_osdmap_epochs": "500", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_complaint_time": "30", 2024-09-19T21:34:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_duration": "600", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_size": "20", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_slow_op_size": "20", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_log_threshold": "5", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_in": "false", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_cache_size": "500", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-19T21:34:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_down_out_interval": "600", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_force_trim_to": "0", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-19T21:34:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_down_reporters": "2", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_report_timeout": "900", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-19T21:34:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_stuck_threshold": "60", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_probe_timeout": "2.000000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_max_change": "0.050000", 2024-09-19T21:34:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_max_osds": "4", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_interval": "86400", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_max_keys": "100", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_timeout": "300", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_session_timeout": "300", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_smart_report_timeout": "5", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stat_smooth_intervals": "6", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_pool_min_size": "2", 2024-09-19T21:34:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_pool_size": "4", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_debug": "false", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_provider_kill_at": "0", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_requester_kill_at": "0", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_timeout": "60.000000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_target_pg_per_osd": "100", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_tick_interval": "5", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_timecheck_interval": "300.000000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_use_min_delay_socket": "false", 2024-09-19T21:34:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_older_version_delay": "604800", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_misplaced": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_older_version": "true", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-19T21:34:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "monmap": "", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-19T21:34:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_tracing_enabled": "false", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_op_threads": "3", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_cm": "false", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_device_name": "", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_dscp": "96", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_local_gid": "", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_port_num": "1", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-19T21:34:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_sl": "3", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_support_srq": "true", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_type": "ib", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_reap_threshold": "5", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_before_connect": "false", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_ipv4": "true", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_ipv6": "false", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_msgr1": "true", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_msgr2": "true", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_port_max": "7568", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_port_min": "6800", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_retry_count": "3", 2024-09-19T21:34:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_retry_delay": "5", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_client": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mds": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mgr": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mon": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_osd": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_client_mode": "crc secure", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_cluster_mode": "crc secure", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_cluster_type": "", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_compress_secure": "false", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_connection_idle_timeout": "900", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_connection_ready_timeout": "10", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_crc_data": "true", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_crc_header": "true", 2024-09-19T21:34:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_bad_msg": "false", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_bug": "true", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_old_message": "true", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_skipped_message": "false", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_coremask": "0xF", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_enable_tso": "true", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hugepages": "", 2024-09-19T21:34:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_lro": "true", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_memory_channel": "4", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_pmd": "", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_port_id": "0", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dump_on_send": "false", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_initial_backoff": "0.200000", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_max": "1.000000", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_type": "", 2024-09-19T21:34:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_network_congestion": "0", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_socket_failures": "5000", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_learn_addr_from_peer": "true", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_max_accept_failures": "4", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_max_backoff": "15.000000", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_client_mode": "secure crc", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_service_mode": "secure crc", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compress_min_size": "1024", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compress_mode": "none", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-19T21:34:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_pq_min_cost": "65536", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_public_type": "", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_service_mode": "crc secure", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_listen_backlog": "512", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_nodelay": "true", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_rcvbuf": "0", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_type": "async+posix", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "no_config_file": "false", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_completion_locks_per_session": "32", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inflight_ops": "1024", 2024-09-19T21:34:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_tick_interval": "5.000000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_timeout": "10.000000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "objectstore_blackhole": "false", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "openssl_engine_opts": "", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_delay_time": "5.000000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_hist_halflife": "1000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_max_low_ops": "2", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_max_ops": "4", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_slop": "0.020000", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-19T21:34:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_asio_thread_count": "2", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_async_recovery_min_cost": "100", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_scan_max": "512", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_scan_min": "64", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_degraded": "false", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_peering": "false", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_unfound": "true", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_beacon_report_interval": "300", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_duration": "30", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-19T21:34:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_max_block_size": "67108864", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_blkin_trace_all": "false", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_check_for_log_corruption": "false", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_update_on_start": "true", 2024-09-19T21:34:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_message_cap": "256", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_message_size_cap": "524288000", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_op_priority": "63", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_watch_timeout": "30", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_command_max_records": "256", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_compact_on_start": "false", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_update_on_start": "true", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_update_weight_set": "true", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-19T21:34:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_feed_pullee": "-1", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_misdirected_ops": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_no_acting_change": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_no_purge_strays": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_op_order": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-19T21:34:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_shutdown": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_snaps": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_keys": "1024", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-19T21:34:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_stride": "524288", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_default_notify_timeout": "30", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep": "0.000000", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_discard_disconnected_ops": "true", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_enable_op_tracker": "true", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-19T21:34:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_info": "true", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown": "true", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_function_tracing": "false", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_grace": "20", 2024-09-19T21:34:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_interval": "6", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_peers": "10", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_size": "2000", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_stale": "600", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_max_size": "100000", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_min_size": "1000", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-19T21:34:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal_size": "5120", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_kill_backfill_at": "0", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_cache_size": "50", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_dedup": "true", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_message_max": "40", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_share_max_epochs": "40", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_attr_name_len": "100", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_attr_size": "0", 2024-09-19T21:34:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_backfills": "1", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_markdown_count": "5", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_markdown_period": "600", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_name_len": "2048", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_namespace_len": "256", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_size": "134217728", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_blocked_by": "16", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_log_entries": "10000", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pgls": "1024", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_push_cost": "8388608", 2024-09-19T21:34:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_push_objects": "10", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_scrubs": "3", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_trimming_pgs": "2", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_write_op_reply_len": "64", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_write_size": "90", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-19T21:34:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_profile": "balanced", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-19T21:34:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_base": "805306368", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_cache_min": "134217728", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target": "4294967296", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target_autotune": "false", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_min_pg_log_entries": "250", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_min_recovery_priority": "0", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_report_interval": "5", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-19T21:34:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_num_cache_shards": "32", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_num_op_tracker_shard": "32", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_auto_affinity": "true", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_node": "-1", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_prefer_iface": "true", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objecter_finishers": "1", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore": "bluestore", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore_fuse": "false", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore_tracing": "false", 2024-09-19T21:34:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_complaint_time": "30.000000", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_duration": "600", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_size": "20", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_slow_op_size": "20", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_log_threshold": "5", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards": "0", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards_hdd": "5", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards_ssd": "8", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-19T21:34:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_pq_min_cost": "65536", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_queue_cut_off": "high", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_thread_timeout": "15", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_open_classes_on_start": "true", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_os_flags": "0", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_peering_op_priority": "255", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_delete_cost": "1048576", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_delete_priority": "5", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-19T21:34:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_trim_max": "10000", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_trim_min": "100", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_crimson": "false", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-19T21:34:28.597 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flags": "0", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_min_size": "0", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pg_num": "32", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pgp_num": "0", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-19T21:34:28.598 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_size": "2", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_type": "replicated", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_push_per_object_cost": "1000", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recover_clone_overlap": "true", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_cost": "20971520", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active": "0", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-19T21:34:28.599 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_single_start": "1", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_op_priority": "3", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_priority": "5", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep": "0.000000", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_repair_during_recovery": "false", 2024-09-19T21:34:28.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_requested_scrub_priority": "120", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_auto_repair": "false", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_begin_hour": "0", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_begin_week_day": "0", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_chunk_max": "25", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_chunk_min": "5", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_cost": "52428800", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_during_recovery": "false", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_end_hour": "0", 2024-09-19T21:34:28.601 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_end_week_day": "0", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_event_cost": "4096", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_invalid_stats": "true", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_max_preemptions": "5", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_priority": "5", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_sleep": "0.000000", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-19T21:34:28.602 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_skip_data_digest": "false", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_smart_report_timeout": "5", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_cost": "1048576", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_priority": "5", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-19T21:34:28.603 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_target_transaction_size": "30", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tracing": "false", 2024-09-19T21:34:28.604 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_use_stale_snap": "false", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "osdc_blkin_trace_all": "false", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_kill_at": "0", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_max_join_drift": "10", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_min": "500", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_min_wait": "0.050000", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_propose_interval": "1.000000", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_max": "500", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_min": "250", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_stash_full_interval": "25", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_trim_max": "500", 2024-09-19T21:34:28.605 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_trim_min": "250", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "perf": "true", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "-", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrv": "", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "public_bind_addr": "-", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "public_network": "", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "public_network_interface": "", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "qat_compressor_enabled": "false", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "qat_compressor_session_max_number": "256", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_mon_op_timeout": "0", 2024-09-19T21:34:28.606 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_osd_op_timeout": "0", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_tracing": "false", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_atime_update_interval": "60", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_balance_parent_reads": "false", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_balance_snap_reads": "false", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blkin_trace_all": "false", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache": "true", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-19T21:34:28.607 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_policy": "writearound", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_size": "33554432", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_clone_copy_on_read": "false", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_compression_hint": "none", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_concurrent_management_ops": "10", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_clone_format": "auto", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_data_pool": "", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_features": "61", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_format": "2", 2024-09-19T21:34:28.608 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_map_options": "", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_order": "22", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_pool": "rbd", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_stripe_count": "0", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_stripe_unit": "0", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_enable_alloc_hint": "true", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_io_scheduler": "simple", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-19T21:34:28.609 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_order": "24", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_pool": "", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_splay_width": "4", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_localize_parent_reads": "false", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_localize_snap_reads": "false", 2024-09-19T21:34:28.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-19T21:34:28.611 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-19T21:34:28.612 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mtime_update_interval": "60", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_non_blocking_aio": "true", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_op_thread_timeout": "60", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_op_threads": "1", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_parent_cache_enabled": "false", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-19T21:34:28.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_plugins": "", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_burst": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_limit": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_exclude_ops": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_burst": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_limit": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-19T21:34:28.614 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_read_from_replica_policy": "default", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-19T21:34:28.615 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_skip_partial_discard": "true", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_tracing": "false", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_validate_names": "true", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_validate_pool": "true", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "restapi_base_url": "", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "restapi_log_level": "", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_acl_grants_max_num": "100", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_admin_entry": "admin", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_backend_store": "rados", 2024-09-19T21:34:28.616 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_barbican_url": "", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_beast_enable_async": "true", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_enabled": "true", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_expiry_interval": "900", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_lru_size": "10000", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_config_store": "rados", 2024-09-19T21:34:28.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_content_length_compat": "false", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_copy_obj_progress": "true", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cors_rules_max_num": "100", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cross_domain_policy": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_addr": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_password": "", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-19T21:34:28.618 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_username": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_require_ssl": "true", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-19T21:34:28.619 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_addr": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_auth": "token", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_namespace": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_prefix": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_token_file": "", 2024-09-19T21:34:28.620 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_buffersize": "524288", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_low_speed_time": "300", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-19T21:34:28.621 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_changes_size": "1000", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_num_shards": "128", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_window": "30", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-19T21:34:28.622 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-19T21:34:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dns_name": "", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dns_s3website_name": "", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dynamic_resharding": "true", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_gc_threads": "true", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_lc_threads": "true", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_ops_log": "false", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_quota_threads": "true", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_static_website": "false", 2024-09-19T21:34:28.624 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_usage_log": "false", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enforce_swift_acls": "true", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_exit_timeout_secs": "120", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_expose_bucket": "false", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_extended_http_attrs": "", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_filter": "none", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_frontends": "beast port=7480", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_deferred": "50", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_objs": "32", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-19T21:34:28.625 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_processor_period": "3600", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_init_timeout": "300", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-19T21:34:28.626 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_domain": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_password": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_password_path": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_project": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_tenant": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_token": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_token_path": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_user": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_api_version": "2", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_domain": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_password": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_project": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-19T21:34:28.627 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_user": "", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_url": "", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_debug_interval": "-1", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_lock_max_time": "90", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_objs": "32", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_rules": "1000", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_worker": "3", 2024-09-19T21:34:28.628 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_thread_delay": "0", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_dnattr": "uid", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_searchfilter": "", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_http_headers": "", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-19T21:34:28.629 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_object_name_utc": "false", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attr_name_len": "0", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attr_size": "0", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_chunk_size": "4194304", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_listing_results": "1000", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_notify_retries": "10", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-19T21:34:28.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_put_param_size": "1048576", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_put_size": "5368709120", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_slo_entries": "1000", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_md_log_max_shards": "64", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_mp_lock_max_time": "600", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-19T21:34:28.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_max_gc": "300", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_num_async_rados_threads": "32", 2024-09-19T21:34:28.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_num_control_oids": "8", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_numa_node": "-1", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_chunk_size": "100", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_gc_interval": "600", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_thread_timeout": "600", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_tracing": "false", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_token": "", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_url": "", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_verify_ssl": "true", 2024-09-19T21:34:28.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_rados": "false", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_socket_path": "", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_push_interval": "2.000000", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_print_continue": "true", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-19T21:34:28.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_tracing": "false", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm": "", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm_id": "", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_region": "", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-19T21:34:28.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_request_uri": "", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_batch_size": "64", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_max_aio": "128", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_num_logs": "16", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_thread_interval": "600", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_resolve_cname": "false", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_run_sync_thread": "true", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-19T21:34:28.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_scheduler_type": "throttler", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_script_uri": "", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_service_provider_name": "", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_client_id": "", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_client_secret": "", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_entry": "sts", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_key": "sts", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-19T21:34:28.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_min_session_duration": "900", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_token_introspection_url": "", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_account_in_url": "false", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_auth_entry": "auth", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_auth_url": "", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_custom_header": "", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_need_stats": "true", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_tenant_name": "", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_token_expiration": "86400", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_url": "", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_url_prefix": "swift", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-19T21:34:28.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_lease_period": "120", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_thread_pool_size": "512", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_comment": "", 2024-09-19T21:34:28.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_createby": "", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_encoding": "", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_flag": "false", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_origin": "", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_tracker": "", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_trust_forwarded_https": "false", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_max_shards": "32", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_max_user_shards": "1", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_use_opa_authz": "false", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-19T21:34:28.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_max_buckets": "1000", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_policies_max_num": "100", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_unique_email": "true", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_verify_ssl": "true", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone": "", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone_id": "", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-19T21:34:28.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup": "", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup_id": "", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgwlc_auto_session_clear": "true", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_block_size": "4096", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_size": "536870912", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-19T21:34:28.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_index_type": "binary_search", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_partition_filters": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_perf": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-19T21:34:28.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "run_dir": "/var/run/ceph", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_block_create": "true", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_cache_lru_size": "67108864", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_default_max_object_size": "16777216", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_device_size": "53687091200", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_capacity": "16", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_main_device_type": "SSD", 2024-09-19T21:34:28.644 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_segment_size": "67108864", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "set_keepcaps": "false", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "setgroup": "", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "setuser": "", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "setuser_match_path": "", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "thp": "false", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "threadpool_default_timeout": "60", 2024-09-19T21:34:28.645 INFO:teuthology.orchestra.run.smithi012.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-19T21:34:28.646 INFO:teuthology.orchestra.run.smithi012.stdout: "throttler_perf_counter": "true" 2024-09-19T21:34:28.646 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:28.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-19T21:34:28.788 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a: {} 2024-09-19T21:34:28.804 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b: {} 2024-09-19T21:34:28.817 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c: {} 2024-09-19T21:34:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:34:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:34:28.820 INFO:teuthology.orchestra.run.smithi012.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:34:28.832 DEBUG:teuthology.orchestra.run.smithi012:> type ceph-objectstore-tool 2024-09-19T21:34:28.839 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-19T21:34:28.839 DEBUG:teuthology.orchestra.run.smithi033:> type ceph-objectstore-tool 2024-09-19T21:34:28.846 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-19T21:34:28.846 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-19T21:34:28.859 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-19T21:34:28.859 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-19T21:34:28.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:34:28.859 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:34:28.860 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-19T21:34:28.860 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.823476 2024-09-19T21:34:28.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.8234763477643953 2024-09-19T21:34:28.860 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-19T21:34:28.861 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-19T21:34:28.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:28.861 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-19T21:34:28.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:28.862 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-19T21:34:28.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:34:28.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-19T21:34:28.888 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:28.886+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:28.904 INFO:teuthology.orchestra.run.smithi122.stdout:mapped writes DISABLED 2024-09-19T21:34:28.904 INFO:teuthology.orchestra.run.smithi122.stdout:mapped reads DISABLED 2024-09-19T21:34:28.905 INFO:teuthology.orchestra.run.smithi122.stdout:Seed set to 3698 2024-09-19T21:34:28.905 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-nbd mode enabled 2024-09-19T21:34:28.962 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:28.958+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:29.023 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:29.023 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:29.023 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:29.024 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:29.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:29.044 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:29.063 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:29.056+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:29.065 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:29.084 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:29.105 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:29.129 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.129 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.139 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:29.140 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:29.164 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:29.156+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:29.166 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:29.257 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.258 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.264 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:29.256+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:29.267 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:29.366 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:29.362+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:29.371 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.371 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:29.467 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:29.462+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:29.485 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.485 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:29.567 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:29.560+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:29.607 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.608 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:29.668 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:29.660+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:29.770 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:29.766+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:29.773 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.773 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:29.871 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:29.864+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:29.886 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:29.887 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:29.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:29.972 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:29.966+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:30.012 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.5 primary-affinity to 0.823476 (8539672) 2024-09-19T21:34:30.012 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:34:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.072 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:30.064+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:30.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:30.173 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:30.170+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:30.197 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.197 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:30.274 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:30.270+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:30.317 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.317 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:30.375 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:30.368+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:30.476 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:30.470+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:30.478 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.478 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:30.577 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:30.574+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:30.596 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.596 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:30.678 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:30.672+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:30.779 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:30.772+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:30.822 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.822 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:30.881 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:30.872+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:30.941 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:30.941 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:30.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:30.982 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:30.978+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:31.067 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:31.067 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:31.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:31.082 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:31.078+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:31.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:31.182 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:31.182 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:31.183 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:31.178+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:31.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:31.284 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:31.276+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:31.328 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:31.337 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:31.337 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:31.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:31.349 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:31.370 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:31.384 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:31.376+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:31.394 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:31.418 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:31.442 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:31.444 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:31.452 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:31.453 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:31.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:31.508 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:31.500+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:31.586 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:31.582+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:31.620 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:20.964894+0000", 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.650931719999999, 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027155300000000001, 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:34:31.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964894+0000", 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964894+0000", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964892+0000", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964904+0000", 2024-09-19T21:34:31.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2278e-05 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964920+0000", 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5495999999999999e-05 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.964937+0000", 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7028999999999999e-05 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965012+0000", 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5059000000000001e-05 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965166+0000", 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015378700000000001 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.625 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:20.965402+0000", 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.650424272, 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024468599999999998, 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965402+0000", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965402+0000", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965401+0000", 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965412+0000", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1430000000000001e-05 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965431+0000", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9415e-05 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965445+0000", 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3573e-05 2024-09-19T21:34:31.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965505+0000", 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0306999999999997e-05 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.965646+0000", 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014109999999999999 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6637976269999992, 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0060163100000000004, 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:31.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952025+0000", 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952046+0000", 2024-09-19T21:34:31.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0786999999999999e-05 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952078+0000", 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2021999999999999e-05 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952102+0000", 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3289999999999999e-05 2024-09-19T21:34:31.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952216+0000", 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000114022 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952223+0000", 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1559999999999998e-06 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955067+0000", 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0028442789999999999 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955205+0000", 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138391 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955372+0000", 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000166167 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.956612+0000", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012408199999999999 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957863+0000", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001250384 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957962+0000", 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9715000000000003e-05 2024-09-19T21:34:31.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.958045+0000", 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2161000000000002e-05 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop_reply(client.4224.0:5 2.2 e20/18 ondisk, result = 0)", 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:21.957723+0000", 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6581029300000001, 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033696600000000001, 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957723+0000", 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957723+0000", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957722+0000", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957730+0000", 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0600000000000008e-06 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957751+0000", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0741999999999999e-05 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957761+0000", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0876999999999999e-05 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957830+0000", 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8509000000000004e-05 2024-09-19T21:34:31.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957854+0000", 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4263000000000001e-05 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.958060+0000", 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000205792 2024-09-19T21:34:31.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.6609773390000004, 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033600710000000001, 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:31.642 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954846+0000", 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954861+0000", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.452e-05 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954883+0000", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359000000000001e-05 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954900+0000", 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7008e-05 2024-09-19T21:34:31.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955053+0000", 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000153311 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955302+0000", 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024876800000000001 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955695+0000", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00039324800000000001 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.957181+0000", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001485105 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958132+0000", 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095181900000000002 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958178+0000", 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5419e-05 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958209+0000", 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0908000000000003e-05 2024-09-19T21:34:31.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop_reply(client.4224.0:7 2.2 e21/18 ondisk, result = 0)", 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:22.958007+0000", 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.6578184650000001, 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020693400000000001, 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:31.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958007+0000", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958007+0000", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958006+0000", 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:31.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958017+0000", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1443e-05 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958044+0000", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6259e-05 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958063+0000", 2024-09-19T21:34:31.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9913000000000001e-05 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958122+0000", 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8677000000000001e-05 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958129+0000", 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4999999999999996e-06 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:31.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958214+0000", 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5782e-05 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:31.653 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:31.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:31.687 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:31.680+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:31.763 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:31.763 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:31.763 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:31.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:31.814 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:31.808+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:31.888 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:31.886+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:31.924 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:31.924 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:31.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:31.989 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:31.984+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:32.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.112869691, 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:34:32.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:34:32.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:34:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.869951649000001, 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011699008, 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 37 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:32.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175721+0000", 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5700000000000003e-06 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175741+0000", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0341999999999999e-05 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175748+0000", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4230000000000002e-06 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185384+0000", 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0096361899999999993 2024-09-19T21:34:32.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185499+0000", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000114843 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185626+0000", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012677599999999999 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.186404+0000", 2024-09-19T21:34:32.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000778658 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187288+0000", 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000883179 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187363+0000", 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5041000000000002e-05 2024-09-19T21:34:32.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187416+0000", 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.782781932000001, 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012081382, 2024-09-19T21:34:32.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 87 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262886+0000", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262994+0000", 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108334 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263011+0000", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6419999999999999e-05 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263023+0000", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2663000000000001e-05 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268657+0000", 2024-09-19T21:34:32.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056333720000000002 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268794+0000", 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000137331 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268985+0000", 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019139100000000001 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.271593+0000", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002608057 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274842+0000", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0032483780000000001 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274910+0000", 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8592999999999999e-05 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274968+0000", 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7710000000000001e-05 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.782631022, 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012663806999999999, 2024-09-19T21:34:32.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 88 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263037+0000", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263047+0000", 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0172e-05 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263101+0000", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4493000000000002e-05 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263115+0000", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3733e-05 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270291+0000", 2024-09-19T21:34:32.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0071757640000000003 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270447+0000", 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000156291 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270753+0000", 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030557000000000002 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.271983+0000", 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001230243 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275563+0000", 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035797340000000002 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275642+0000", 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9558e-05 2024-09-19T21:34:32.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275701+0000", 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9138999999999997e-05 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572918295000001, 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.01141986, 2024-09-19T21:34:32.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 112 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472889+0000", 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000139197 2024-09-19T21:34:32.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472909+0000", 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9649000000000001e-05 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472917+0000", 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7439999999999996e-06 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477287+0000", 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0043697110000000001 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477358+0000", 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0549000000000005e-05 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477484+0000", 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000126119 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482197+0000", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047129190000000003 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484110+0000", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001913503 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484142+0000", 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2072000000000001e-05 2024-09-19T21:34:32.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484170+0000", 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8073999999999999e-05 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572742096000001, 2024-09-19T21:34:32.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011770846999999999, 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 113 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472926+0000", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472934+0000", 2024-09-19T21:34:32.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6570000000000003e-06 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472946+0000", 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1365999999999999e-05 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472953+0000", 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0299999999999996e-06 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478126+0000", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051735970000000003 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478206+0000", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9552e-05 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478320+0000", 2024-09-19T21:34:32.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011373400000000001 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482477+0000", 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0041574489999999997 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484576+0000", 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0020983210000000002 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484626+0000", 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0322999999999999e-05 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484697+0000", 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1607999999999996e-05 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.090 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572704027, 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012975662000000001, 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 114 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472964+0000", 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472970+0000", 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4369999999999998e-06 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472979+0000", 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8800000000000003e-06 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472986+0000", 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9859999999999997e-06 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478857+0000", 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0058707689999999996 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478929+0000", 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1633999999999999e-05 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479048+0000", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119163 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482618+0000", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003570045 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485769+0000", 2024-09-19T21:34:32.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031512670000000001 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485865+0000", 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6063999999999994e-05 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485940+0000", 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5030000000000005e-05 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572302896, 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012803063, 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 115 2024-09-19T21:34:32.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473364+0000", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473518+0000", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000153263 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473549+0000", 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0945999999999998e-05 2024-09-19T21:34:32.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473559+0000", 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0246e-05 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479723+0000", 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0061640319999999998 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479860+0000", 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013705100000000001 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.480132+0000", 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027181599999999997 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482903+0000", 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002771187 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486041+0000", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003138278 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486115+0000", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3968999999999996e-05 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486169+0000", 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3767999999999997e-05 2024-09-19T21:34:32.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572092247, 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012816342999999999, 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 116 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473581+0000", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8329999999999999e-06 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473612+0000", 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0009e-05 2024-09-19T21:34:32.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473620+0000", 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5469999999999997e-06 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481072+0000", 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0074516890000000001 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481221+0000", 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149073 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481419+0000", 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000198407 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482910+0000", 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014911460000000001 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486256+0000", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00334557 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486309+0000", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3047000000000002e-05 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486393+0000", 2024-09-19T21:34:32.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3821999999999998e-05 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.572033905, 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014549436000000001, 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.109 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 117 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473634+0000", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473640+0000", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6699999999999999e-06 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473650+0000", 2024-09-19T21:34:32.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0266e-05 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473662+0000", 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.146e-05 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483033+0000", 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00937114 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483198+0000", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016553 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483387+0000", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018844000000000001 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484203+0000", 2024-09-19T21:34:32.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00081628699999999996 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488045+0000", 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0038421129999999999 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488104+0000", 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8486000000000002e-05 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488184+0000", 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0704000000000005e-05 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.13077408, 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014174482, 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.115 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 132 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914894+0000", 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914901+0000", 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5900000000000002e-06 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914911+0000", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7229999999999999e-06 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914918+0000", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8360000000000003e-06 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918602+0000", 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003684091 2024-09-19T21:34:32.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918752+0000", 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149796 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918958+0000", 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020586899999999999 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923728+0000", 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047697879999999996 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928934+0000", 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052066279999999996 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929014+0000", 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9725e-05 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929069+0000", 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5245999999999999e-05 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130557165000001, 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014071554, 2024-09-19T21:34:32.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 133 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915110+0000", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915118+0000", 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8299999999999996e-06 2024-09-19T21:34:32.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915132+0000", 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4103e-05 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915145+0000", 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2096000000000001e-05 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920244+0000", 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050993480000000001 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920394+0000", 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000150517 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920552+0000", 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015807700000000001 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923918+0000", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033651670000000001 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929084+0000", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051667390000000001 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929141+0000", 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6990999999999997e-05 2024-09-19T21:34:32.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929183+0000", 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1736000000000002e-05 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130502216, 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015859873, 2024-09-19T21:34:32.127 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 134 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915336+0000", 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000170736 2024-09-19T21:34:32.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915372+0000", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5828999999999999e-05 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915382+0000", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8300000000000008e-06 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921117+0000", 2024-09-19T21:34:32.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057345859999999999 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921363+0000", 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024658200000000002 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921710+0000", 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034673199999999999 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928915+0000", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0072048800000000003 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.930772+0000", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001857214 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.930906+0000", 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013437 2024-09-19T21:34:32.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931026+0000", 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012006399999999999 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130274328, 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015795664000000001, 2024-09-19T21:34:32.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 135 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915401+0000", 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0600000000000002e-06 2024-09-19T21:34:32.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915410+0000", 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3670000000000003e-06 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915420+0000", 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0396e-05 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923006+0000", 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0075852269999999999 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923081+0000", 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5186999999999997e-05 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923194+0000", 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113071 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929854+0000", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0066603610000000001 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931048+0000", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011941090000000001 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931125+0000", 2024-09-19T21:34:32.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6553999999999999e-05 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931190+0000", 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5129000000000006e-05 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:32.139 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130241129, 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015919566, 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 136 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:32.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915427+0000", 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915433+0000", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.993e-06 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915441+0000", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915458+0000", 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6820000000000002e-05 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923948+0000", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0084897940000000002 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.924079+0000", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130877 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.924265+0000", 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185899 2024-09-19T21:34:32.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929920+0000", 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056554149999999996 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931211+0000", 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012910140000000001 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931284+0000", 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3077999999999994e-05 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931347+0000", 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2996000000000005e-05 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.145 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130200092999999, 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016022577999999999, 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 137 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915468+0000", 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915623+0000", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015469600000000001 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915640+0000", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7387000000000001e-05 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915652+0000", 2024-09-19T21:34:32.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2112e-05 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925179+0000", 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0095269759999999995 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925263+0000", 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3497000000000004e-05 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925373+0000", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110048 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931369+0000", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0059966840000000004 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931425+0000", 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5809e-05 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931467+0000", 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2107999999999997e-05 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931491+0000", 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3924e-05 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.151 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.130006743999999, 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015929169999999999, 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 138 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915661+0000", 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915668+0000", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915692+0000", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4352999999999999e-05 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915701+0000", 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6270000000000006e-06 2024-09-19T21:34:32.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926019+0000", 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010317941000000001 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926090+0000", 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1846000000000003e-05 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926193+0000", 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102983 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931384+0000", 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051904070000000002 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931533+0000", 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014889499999999999 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931569+0000", 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6578000000000001e-05 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931591+0000", 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1863999999999999e-05 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.157 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.129893790000001, 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017414302, 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 139 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915774+0000", 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915783+0000", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7129999999999994e-06 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915792+0000", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2129999999999997e-06 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915803+0000", 2024-09-19T21:34:32.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0793e-05 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926692+0000", 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010888616 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926828+0000", 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013679100000000001 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.927017+0000", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000188729 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931628+0000", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0046104650000000002 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933068+0000", 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001440167 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933149+0000", 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1642000000000002e-05 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933189+0000", 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9805999999999998e-05 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.129854916999999, 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017585809000000001, 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 140 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915813+0000", 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915954+0000", 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014095000000000001 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915964+0000", 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.039e-05 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915972+0000", 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928045+0000", 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012073599000000001 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928122+0000", 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6210000000000004e-05 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928231+0000", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109438 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931841+0000", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036096019999999999 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933200+0000", 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013596280000000001 2024-09-19T21:34:32.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933237+0000", 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6671000000000002e-05 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933400+0000", 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000162652 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.129688558, 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017506562, 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 141 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.170 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915979+0000", 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915989+0000", 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915997+0000", 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2360000000000004e-06 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.916004+0000", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6930000000000001e-06 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929219+0000", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013214755 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929299+0000", 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0203000000000004e-05 2024-09-19T21:34:32.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929408+0000", 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108924 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.932599+0000", 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003191138 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:32.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933417+0000", 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00081806700000000001 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933461+0000", 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4462999999999998e-05 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933487+0000", 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5216e-05 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.176 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:32.177 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:32.178 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:32.174+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:32.278 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:32.274+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:32.329 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:32.329 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:32.329 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:32.329 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:32.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:32.379 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:32.374+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:32.442 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:32.443 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:32.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:32.480 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:32.478+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:32.563 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4224.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.604509533, 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0054848090000000002, 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:32.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955032+0000", 2024-09-19T21:34:32.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:32.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955040+0000", 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8399999999999995e-06 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955053+0000", 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2928000000000001e-05 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955067+0000", 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3743999999999999e-05 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955149+0000", 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2071999999999996e-05 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955180+0000", 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0929000000000002e-05 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.960458+0000", 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052780129999999998 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.960517+0000", 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9728000000000001e-05 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4358.0:12 3.1 e26/23 3:85d48609:::rbd_header.11068c3bf21f:head v 26'1, mlcod=0'0)", 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.020813+0000", 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.53872906799999998, 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012295439999999999, 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020813+0000", 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020813+0000", 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020813+0000", 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020818+0000", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5369999999999997e-06 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020837+0000", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8396000000000001e-05 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020848+0000", 2024-09-19T21:34:32.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1093e-05 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020916+0000", 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8428000000000003e-05 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.020944+0000", 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7188000000000001e-05 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.021944+0000", 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010004429999999999 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022043+0000", 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8652000000000003e-05 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4358.0:13 3.3 e26/23 3:c590f66e:::rbd_object_map.11068c3bf21f:head v 26'1, mlcod=0'0)", 2024-09-19T21:34:32.576 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.023456+0000", 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.53608592300000002, 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001392138, 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023456+0000", 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023456+0000", 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023456+0000", 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023464+0000", 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6770000000000001e-06 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023478+0000", 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3786e-05 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023488+0000", 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0402e-05 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023558+0000", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9457999999999998e-05 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023582+0000", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4474999999999999e-05 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.024805+0000", 2024-09-19T21:34:32.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012231379999999999 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.024848+0000", 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2929000000000002e-05 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4358.0:25 3.1 e26/23 3:85d48609:::rbd_header.11068c3bf21f:head v 26'2, mlcod=0'0)", 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.042970+0000", 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.51657252399999998, 2024-09-19T21:34:32.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001231794, 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.042970+0000", 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.042970+0000", 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.042968+0000", 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.042985+0000", 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7136e-05 2024-09-19T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.043016+0000", 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0741000000000003e-05 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.043031+0000", 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5262000000000001e-05 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.043107+0000", 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5992e-05 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.043148+0000", 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0954000000000001e-05 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.044131+0000", 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00098305799999999998 2024-09-19T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.044201+0000", 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0482999999999996e-05 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4470.0:12 3.1 e26/23 3:85d48609:::rbd_header.11068c3bf21f:head v 26'3, mlcod=26'1)", 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.160674+0000", 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.39886774000000003, 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013569369999999999, 2024-09-19T21:34:32.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160674+0000", 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160674+0000", 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160673+0000", 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160687+0000", 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4696e-05 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160712+0000", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4406e-05 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160724+0000", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2136e-05 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160812+0000", 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7603999999999997e-05 2024-09-19T21:34:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.160869+0000", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7107000000000001e-05 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.161979+0000", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011102799999999999 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.162031+0000", 2024-09-19T21:34:32.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2444999999999997e-05 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:32.591 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:32.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:32.593 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:32.584+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:32.693 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:32.690+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:32.712 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:32.713 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:32.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:32.794 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:32.790+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:32.830 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:32.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:32.895 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:32.890+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [] 2024-09-19T21:34:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:32.996 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:32.994+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:33.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:33.032 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:34:33.097 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:33.094+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:33.114 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:33.115 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:33.115 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:33.115 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:33.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:33.198 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:33.194+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:33.279 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:33.279 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:33.279 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:33.279 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:33.279 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:33.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:33.299 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:33.292+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:33.400 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:33.392+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4224.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-19T21:34:33.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:21.958309+0000", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.439640093, 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019343920000000001, 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958309+0000", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958309+0000", 2024-09-19T21:34:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958308+0000", 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958317+0000", 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9099999999999994e-06 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958351+0000", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3908999999999997e-05 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958393+0000", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1783000000000003e-05 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958491+0000", 2024-09-19T21:34:33.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7757000000000006e-05 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.958543+0000", 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2575999999999999e-05 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.960159+0000", 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016158819999999999 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:21.960243+0000", 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4067999999999995e-05 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4224.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:22.958724+0000", 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.43922521, 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018352799999999999, 2024-09-19T21:34:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958724+0000", 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958724+0000", 2024-09-19T21:34:33.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958723+0000", 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958735+0000", 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.163e-05 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958783+0000", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8971999999999998e-05 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.958866+0000", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2378000000000003e-05 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.959023+0000", 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015732699999999999 2024-09-19T21:34:33.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.959072+0000", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8862000000000001e-05 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.960476+0000", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014042530000000001 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:22.960559+0000", 2024-09-19T21:34:33.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2528000000000007e-05 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:12 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.020126+0000", 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3778229900000001, 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050627449999999996, 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 12 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020126+0000", 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020126+0000", 2024-09-19T21:34:33.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020124+0000", 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020138+0000", 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4371e-05 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020154+0000", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6464e-05 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020169+0000", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4352000000000001e-05 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020256+0000", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7120000000000006e-05 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-19T21:34:33.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.020262+0000", 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5520000000000001e-06 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.022708+0000", 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024459009999999999 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.022903+0000", 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019517499999999999 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.023246+0000", 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034256800000000002 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.024571+0000", 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001325136 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.025095+0000", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052377399999999996 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.025147+0000", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1974000000000001e-05 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.025189+0000", 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1612000000000001e-05 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:16 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.035604+0000", 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3623449810000001, 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00045671699999999998, 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 16 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035604+0000", 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035604+0000", 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035602+0000", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035617+0000", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5092000000000001e-05 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035638+0000", 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0285999999999999e-05 2024-09-19T21:34:33.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035651+0000", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3271000000000001e-05 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035756+0000", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105444 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.035842+0000", 2024-09-19T21:34:33.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5667000000000005e-05 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036061+0000", 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021856499999999999 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:17 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.036498+0000", 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.361451215, 2024-09-19T21:34:33.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037517399999999998, 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036498+0000", 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036498+0000", 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036492+0000", 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036506+0000", 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3973e-05 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036532+0000", 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5969000000000002e-05 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036542+0000", 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0268000000000001e-05 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036608+0000", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6095999999999999e-05 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036722+0000", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113527 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.036873+0000", 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015085899999999999 2024-09-19T21:34:33.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:18 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.037313+0000", 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3606356150000001, 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050476299999999998, 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037313+0000", 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037313+0000", 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037312+0000", 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037323+0000", 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1144999999999999e-05 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037342+0000", 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8907999999999999e-05 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037351+0000", 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9439999999999994e-06 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037425+0000", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.462e-05 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037563+0000", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013807899999999999 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.037818+0000", 2024-09-19T21:34:33.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000254565 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:20 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.039615+0000", 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3583335189999999, 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00069633300000000004, 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 20 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039615+0000", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039615+0000", 2024-09-19T21:34:33.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039614+0000", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039623+0000", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1409999999999994e-06 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039642+0000", 2024-09-19T21:34:33.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9131999999999999e-05 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039663+0000", 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0937999999999998e-05 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039738+0000", 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4431000000000006e-05 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.039844+0000", 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106803 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040312+0000", 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046725100000000002 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:21 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.040788+0000", 2024-09-19T21:34:33.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3571603919999999, 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076940199999999998, 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 21 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040788+0000", 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040788+0000", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040787+0000", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040798+0000", 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0788000000000001e-05 2024-09-19T21:34:33.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040829+0000", 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1244000000000001e-05 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040838+0000", 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2289999999999992e-06 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.040936+0000", 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7491999999999994e-05 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.041082+0000", 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014645700000000001 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.041558+0000", 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00047564900000000003 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.042045+0000", 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3559042219999999, 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041131700000000002, 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042045+0000", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042045+0000", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042043+0000", 2024-09-19T21:34:33.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042065+0000", 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1929999999999998e-05 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042085+0000", 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9799000000000001e-05 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042094+0000", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8419999999999994e-06 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042168+0000", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4739000000000004e-05 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042282+0000", 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011328 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.042456+0000", 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017415900000000001 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:24 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.044088+0000", 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3538607030000001, 2024-09-19T21:34:33.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037600699999999998, 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 24 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044088+0000", 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044088+0000", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044086+0000", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044097+0000", 2024-09-19T21:34:33.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0893000000000001e-05 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044122+0000", 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4566000000000002e-05 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044134+0000", 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1888999999999999e-05 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044210+0000", 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6118999999999994e-05 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044307+0000", 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7328000000000001e-05 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.044464+0000", 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000156799 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.450 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:25 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [watch watch cookie 139809104344992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.045085+0000", 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.3528636000000001, 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002304768, 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 25 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045085+0000", 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045085+0000", 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045079+0000", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999938 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045094+0000", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5631999999999999e-05 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045120+0000", 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6077000000000001e-05 2024-09-19T21:34:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045150+0000", 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9774e-05 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045242+0000", 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1834000000000002e-05 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045359+0000", 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011652199999999999 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.045536+0000", 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017758800000000001 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.046671+0000", 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011349820000000001 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.047212+0000", 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00054065700000000005 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.047273+0000", 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1575999999999995e-05 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.047390+0000", 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116531 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:3 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.152363+0000", 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.245585532, 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000716795, 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152363+0000", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152363+0000", 2024-09-19T21:34:33.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152361+0000", 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152375+0000", 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3720000000000001e-05 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152400+0000", 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5272000000000001e-05 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152422+0000", 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2073e-05 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152547+0000", 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012442400000000001 2024-09-19T21:34:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.152706+0000", 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000159207 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153080+0000", 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00037418500000000002 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:4 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.153352+0000", 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2445968590000001, 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000461569, 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153352+0000", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153352+0000", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153350+0000", 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153362+0000", 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1979e-05 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153386+0000", 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4097e-05 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153406+0000", 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9859000000000001e-05 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153508+0000", 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135400000000001 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153639+0000", 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000131304 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.153813+0000", 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017463299999999999 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:5 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.154126+0000", 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.243822416, 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058133999999999996, 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 5 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154126+0000", 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154126+0000", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154125+0000", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154135+0000", 2024-09-19T21:34:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0416e-05 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154158+0000", 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3246000000000001e-05 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154177+0000", 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8372999999999998e-05 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154274+0000", 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7357999999999999e-05 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154428+0000", 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015378000000000001 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.154708+0000", 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027989699999999998 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:6 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.155003+0000", 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2429457989999999, 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00048438600000000001, 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 6 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.471 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155003+0000", 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155003+0000", 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155001+0000", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155011+0000", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0103e-05 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155034+0000", 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2753000000000001e-05 2024-09-19T21:34:33.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155053+0000", 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9059000000000002e-05 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155154+0000", 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100861 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155288+0000", 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000133604 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.155487+0000", 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019964900000000001 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:7 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.156474+0000", 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.241474414, 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00093516200000000002, 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 7 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156474+0000", 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156474+0000", 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156473+0000", 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156482+0000", 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2900000000000008e-06 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156505+0000", 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2946e-05 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156523+0000", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7329e-05 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156650+0000", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012712499999999999 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.156807+0000", 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015691899999999999 2024-09-19T21:34:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157410+0000", 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00060289999999999996 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:8 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.157869+0000", 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.240080088, 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00071650000000000001, 2024-09-19T21:34:33.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157869+0000", 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157869+0000", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157867+0000", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157877+0000", 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8740000000000007e-06 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157900+0000", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2821999999999999e-05 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.157917+0000", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6977e-05 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.158016+0000", 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8738000000000002e-05 2024-09-19T21:34:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.158152+0000", 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013619700000000001 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.158585+0000", 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043336600000000002 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:11 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.161815+0000", 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2361341109999999, 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049769400000000002, 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 11 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161815+0000", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161815+0000", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161813+0000", 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161837+0000", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3776000000000001e-05 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161872+0000", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5076000000000001e-05 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.161894+0000", 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1872999999999999e-05 2024-09-19T21:34:33.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162022+0000", 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012805299999999999 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162172+0000", 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015064500000000001 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162312+0000", 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014015100000000001 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:12 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [watch watch cookie 139786622953712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.162830+0000", 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.235118527, 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023324840000000001, 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 12 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162830+0000", 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162830+0000", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162829+0000", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162838+0000", 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4800000000000007e-06 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162865+0000", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6709000000000001e-05 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162884+0000", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8499999999999999e-05 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.162966+0000", 2024-09-19T21:34:33.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2618000000000001e-05 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.163047+0000", 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0450000000000004e-05 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.163217+0000", 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016995000000000001 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.164376+0000", 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011599030000000001 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.165037+0000", 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00066038700000000002 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.165094+0000", 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6984999999999997e-05 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.165163+0000", 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8905000000000005e-05 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4470.0:14 3.1 3:85d48609:::rbd_header.11068c3bf21f:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:32.739627+0000", 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.658321924, 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063696, 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:33.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4470", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 14 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739627+0000", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739627+0000", 2024-09-19T21:34:33.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739624+0000", 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739644+0000", 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0573000000000001e-05 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739709+0000", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4797999999999999e-05 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739728+0000", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8346999999999999e-05 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739821+0000", 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3016000000000005e-05 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.739988+0000", 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016750399999999999 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:32.740264+0000", 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027574199999999999 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:33.499 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:33.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:33.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:33.502 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:33.498+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:33.602 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:33.598+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:33.654 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:33.655 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:33.656 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:33.656 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:33.656 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:33.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:33.674 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:33.695 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:33.703 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:33.696+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:33.715 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:33.737 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:33.762 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:33.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:33.772 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:33.772 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:33.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:33.804 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:33.802+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:33.890 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:33.891 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:33.900 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:33.905 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:33.896+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:34.006 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:34.000+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:34.020 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.021 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:34.107 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:34.100+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:34.189 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.189 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:34.207 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:34.200+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:34.304 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.304 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.308 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:34.306+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:34.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:34.409 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:34.406+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:34.471 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.471 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.482 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:34.510 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:34.506+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:34.588 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.588 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:34.642 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:34.636+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:34.706 INFO:teuthology.orchestra.run.smithi122.stdout:2 write 0x1185600 thru 0x1189a62 (0x4463 bytes) 2024-09-19T21:34:34.708 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.708 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.712 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:34.704+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:34.718 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:34.760 INFO:teuthology.orchestra.run.smithi122.stdout:3 write 0xaa45600 thru 0xaa4f736 (0xa137 bytes) 2024-09-19T21:34:34.797 INFO:teuthology.orchestra.run.smithi122.stdout:6 write 0xb13a600 thru 0xb13dc74 (0x3675 bytes) 2024-09-19T21:34:34.799 INFO:teuthology.orchestra.run.smithi122.stdout:7 read 0x4703e00 thru 0x4709ab3 (0x5cb4 bytes) 2024-09-19T21:34:34.799 INFO:teuthology.orchestra.run.smithi122.stdout:10 read 0x4160800 thru 0x41676bd (0x6ebe bytes) 2024-09-19T21:34:34.799 INFO:teuthology.orchestra.run.smithi122.stdout:13 punch from 0xa72e000 to 0xa739600, (0xb600 bytes) 2024-09-19T21:34:34.800 INFO:teuthology.orchestra.run.smithi122.stdout:15 write 0xa85fc00 thru 0xa861e64 (0x2265 bytes) 2024-09-19T21:34:34.802 INFO:teuthology.orchestra.run.smithi122.stdout:16 read 0x6882c00 thru 0x6887317 (0x4718 bytes) 2024-09-19T21:34:34.802 INFO:teuthology.orchestra.run.smithi122.stdout:20 write 0x2229000 thru 0x223510a (0xc10b bytes) 2024-09-19T21:34:34.803 INFO:teuthology.orchestra.run.smithi122.stdout:22 read 0x4192e00 thru 0x419a76f (0x7970 bytes) 2024-09-19T21:34:34.803 INFO:teuthology.orchestra.run.smithi122.stdout:29 read 0x279b800 thru 0x27a042d (0x4c2e bytes) 2024-09-19T21:34:34.803 INFO:teuthology.orchestra.run.smithi122.stdout:30 read 0x3579e00 thru 0x35830e8 (0x92e9 bytes) 2024-09-19T21:34:34.803 INFO:teuthology.orchestra.run.smithi122.stdout:32 punch from 0x86ed000 to 0x86f3a00, (0x6a00 bytes) 2024-09-19T21:34:34.804 INFO:teuthology.orchestra.run.smithi122.stdout:33 read 0x3a18600 thru 0x3a25714 (0xd115 bytes) 2024-09-19T21:34:34.805 INFO:teuthology.orchestra.run.smithi122.stdout:35 write 0x7266200 thru 0x726a87e (0x467f bytes) 2024-09-19T21:34:34.805 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0x498cc00 2024-09-19T21:34:34.813 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:34.810+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:34.818 INFO:teuthology.orchestra.run.smithi122.stdout:37 trunc from 0xb13de00 to 0x498cc00 2024-09-19T21:34:34.830 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.830 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:34.855 INFO:teuthology.orchestra.run.smithi122.stdout:38 write 0x2030200 thru 0x203c70a (0xc50b bytes) 2024-09-19T21:34:34.857 INFO:teuthology.orchestra.run.smithi122.stdout:40 punch from 0x25fec00 to 0x260d800, (0xec00 bytes) 2024-09-19T21:34:34.858 INFO:teuthology.orchestra.run.smithi122.stdout:41 read 0x31d4a00 thru 0x31e2e47 (0xe448 bytes) 2024-09-19T21:34:34.882 INFO:teuthology.orchestra.run.smithi122.stdout:44 write 0x529d800 thru 0x52ad5a1 (0xfda2 bytes) 2024-09-19T21:34:34.886 INFO:teuthology.orchestra.run.smithi122.stdout:45 trunc from 0x52ad600 to 0x45ace00 2024-09-19T21:34:34.914 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:34.908+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:34.945 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:34.945 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:34.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:34.960 INFO:teuthology.orchestra.run.smithi122.stdout:48 write 0x81a4600 thru 0x81af2b7 (0xacb8 bytes) 2024-09-19T21:34:34.962 INFO:teuthology.orchestra.run.smithi122.stdout:49 punch from 0x127f600 to 0x1288400, (0x8e00 bytes) 2024-09-19T21:34:34.964 INFO:teuthology.orchestra.run.smithi122.stdout:51 punch from 0x2a2a000 to 0x2a33800, (0x9800 bytes) 2024-09-19T21:34:34.964 INFO:teuthology.orchestra.run.smithi122.stdout:52 read 0x11ed600 thru 0x11f2c10 (0x5611 bytes) 2024-09-19T21:34:34.968 INFO:teuthology.orchestra.run.smithi122.stdout:58 punch from 0x61ce00 to 0x621400, (0x4600 bytes) 2024-09-19T21:34:34.968 INFO:teuthology.orchestra.run.smithi122.stdout:61 punch from 0x18c9200 to 0x18d4a00, (0xb800 bytes) 2024-09-19T21:34:34.968 INFO:teuthology.orchestra.run.smithi122.stdout:62 write 0x46e8000 thru 0x46e8329 (0x32a bytes) 2024-09-19T21:34:34.969 INFO:teuthology.orchestra.run.smithi122.stdout:63 write 0x1705e00 thru 0x1710719 (0xa91a bytes) 2024-09-19T21:34:34.969 INFO:teuthology.orchestra.run.smithi122.stdout:64 read 0xca1000 thru 0xca5408 (0x4409 bytes) 2024-09-19T21:34:34.998 INFO:teuthology.orchestra.run.smithi122.stdout:65 write 0x81a1800 thru 0x81b0c8a (0xf48b bytes) 2024-09-19T21:34:35.016 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:35.014+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:35.018 INFO:teuthology.orchestra.run.smithi122.stdout:67 trunc from 0x81b0e00 to 0xdc1e00 2024-09-19T21:34:35.024 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:34:35.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-19T21:34:35.047 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:34:35.083 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.083 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.092 INFO:teuthology.orchestra.run.smithi122.stdout:69 write 0x8119600 thru 0x811b2eb (0x1cec bytes) 2024-09-19T21:34:35.096 INFO:teuthology.orchestra.run.smithi122.stdout:70 write 0x5773c00 thru 0x5782580 (0xe981 bytes) 2024-09-19T21:34:35.096 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0x96abe00 2024-09-19T21:34:35.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:35.099 INFO:teuthology.orchestra.run.smithi122.stdout:74 trunc from 0x811b400 to 0x96abe00 2024-09-19T21:34:35.129 INFO:teuthology.orchestra.run.smithi122.stdout:77 punch from 0x335b000 to 0x3365000, (0xa000 bytes) 2024-09-19T21:34:35.130 INFO:teuthology.orchestra.run.smithi122.stdout:80 read 0x3b4bc00 thru 0x3b551b4 (0x95b5 bytes) 2024-09-19T21:34:35.144 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:35.142+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:35.158 INFO:teuthology.orchestra.run.smithi122.stdout:81 write 0xcf11400 thru 0xcf16ad4 (0x56d5 bytes) 2024-09-19T21:34:35.183 INFO:teuthology.orchestra.run.smithi122.stdout:84 trunc from 0xcf16c00 to 0x1547a00 2024-09-19T21:34:35.217 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:35.208+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:35.221 INFO:teuthology.orchestra.run.smithi122.stdout:86 read 0xb87a00 thru 0xb8fd48 (0x8349 bytes) 2024-09-19T21:34:35.223 INFO:teuthology.orchestra.run.smithi122.stdout:87 read 0xc3a600 thru 0xc4808f (0xda90 bytes) 2024-09-19T21:34:35.226 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.226 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:35.265 INFO:teuthology.orchestra.run.smithi122.stdout:88 write 0xb354800 thru 0xb35f89c (0xb09d bytes) 2024-09-19T21:34:35.267 INFO:teuthology.orchestra.run.smithi122.stdout:89 write 0x2130c00 thru 0x2133d47 (0x3148 bytes) 2024-09-19T21:34:35.267 INFO:teuthology.orchestra.run.smithi122.stdout:90 clone 1 order 19 su 4096 sc 2 2024-09-19T21:34:35.318 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:35.314+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:35.349 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.349 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:35.419 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:35.412+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:35.468 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.468 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:35.519 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:35.514+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:35.582 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.582 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:35.620 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:35.618+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:35.702 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.703 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:35.721 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:35.718+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:35.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:35.822 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.822 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.826 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:35.820+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:35.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:35.923 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:35.918+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:35.929 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:35.952 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:35.971 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:35.990 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:35.990 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:35.991 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:36.000 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:36.014 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:36.039 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:36.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:36.044 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:36.044 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:34:36.048 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:36.046+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:36.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-19T21:34:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:36.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:36.125 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:36.122+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:36.225 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:36.216+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-09-19T21:34:36.233 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.270671+0000", 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.958500374, 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046376899999999999, 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270671+0000", 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270671+0000", 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270669+0000", 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270683+0000", 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3368e-05 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270711+0000", 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8547999999999998e-05 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270723+0000", 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1831000000000001e-05 2024-09-19T21:34:36.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270803+0000", 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0143999999999997e-05 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.270834+0000", 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0852000000000002e-05 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.275246+0000", 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0044115509999999997 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.275309+0000", 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3015999999999994e-05 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:87 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'52)", 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.271855+0000", 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.957316295999998, 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.005337751, 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271855+0000", 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271855+0000", 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271853+0000", 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271924+0000", 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1231999999999998e-05 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271943+0000", 2024-09-19T21:34:36.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9099000000000001e-05 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.271956+0000", 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.204e-05 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.272036+0000", 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0162999999999998e-05 2024-09-19T21:34:36.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.272066+0000", 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.023e-05 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.277132+0000", 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050655989999999996 2024-09-19T21:34:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.277193+0000", 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1321000000000005e-05 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:104 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-09-19T21:34:36.244 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.754610143000001, 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077086730000000001, 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474560+0000", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474577+0000", 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7283e-05 2024-09-19T21:34:36.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474609+0000", 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2316e-05 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474622+0000", 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.305e-05 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474765+0000", 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014250999999999999 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474825+0000", 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0529000000000003e-05 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482210+0000", 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073848660000000003 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482270+0000", 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9786e-05 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.753416939000001, 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006550213, 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:36.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475753+0000", 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475771+0000", 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8067000000000002e-05 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475808+0000", 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6665000000000002e-05 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475825+0000", 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7827000000000001e-05 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475942+0000", 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011702 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475984+0000", 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1585999999999998e-05 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482279+0000", 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062948850000000001 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482305+0000", 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5786999999999999e-05 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.753224580000001, 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087792770000000003, 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475945+0000", 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476124+0000", 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000178949 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476158+0000", 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3200000000000001e-05 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476181+0000", 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3898999999999999e-05 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476764+0000", 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058244000000000004 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476831+0000", 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6792999999999996e-05 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484666+0000", 2024-09-19T21:34:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078356450000000005 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484726+0000", 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9701000000000003e-05 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.752419791000001, 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080060370000000006, 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:36.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476750+0000", 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476768+0000", 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8566000000000001e-05 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476817+0000", 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8523000000000003e-05 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476833+0000", 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6079999999999999e-05 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477864+0000", 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001031243 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477897+0000", 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3163000000000001e-05 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484733+0000", 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068351740000000003 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484758+0000", 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5075e-05 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.751837507000001, 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007490137, 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477332+0000", 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477346+0000", 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3219e-05 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477374+0000", 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8799999999999999e-05 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477388+0000", 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3129000000000001e-05 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478476+0000", 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010886030000000001 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478507+0000", 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1083e-05 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484770+0000", 2024-09-19T21:34:36.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062626540000000003 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484824+0000", 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4151999999999999e-05 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.751188981999999, 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068625409999999998, 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477981+0000", 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478037+0000", 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6742e-05 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478068+0000", 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0303000000000001e-05 2024-09-19T21:34:36.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478094+0000", 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5925999999999999e-05 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479202+0000", 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011083219999999999 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479240+0000", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7592999999999997e-05 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484826+0000", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055858959999999999 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484845+0000", 2024-09-19T21:34:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9425000000000001e-05 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.750195832999999, 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066455869999999997, 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.277 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478974+0000", 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478988+0000", 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4202999999999999e-05 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479013+0000", 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978999999999999e-05 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479036+0000", 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2959000000000002e-05 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480591+0000", 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015545730000000001 2024-09-19T21:34:36.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480637+0000", 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5939000000000002e-05 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485516+0000", 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004879899 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485621+0000", 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104715 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:111 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'64)", 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.479624+0000", 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.749546998, 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060677939999999996, 2024-09-19T21:34:36.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479624+0000", 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479624+0000", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479623+0000", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479635+0000", 2024-09-19T21:34:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.224e-05 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479666+0000", 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0982000000000003e-05 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479696+0000", 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9807000000000001e-05 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481693+0000", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019965790000000001 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481735+0000", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2477e-05 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485634+0000", 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0038986009999999998 2024-09-19T21:34:36.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485692+0000", 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8511000000000003e-05 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:112 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'64)", 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.480329+0000", 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.748842580000002, 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053858120000000002, 2024-09-19T21:34:36.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480329+0000", 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480329+0000", 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480327+0000", 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480453+0000", 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012615799999999999 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480482+0000", 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9536e-05 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480500+0000", 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7635999999999999e-05 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482472+0000", 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001971577 2024-09-19T21:34:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482570+0000", 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8324000000000003e-05 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485694+0000", 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003124085 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485715+0000", 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0585999999999999e-05 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:113 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'64)", 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.481071+0000", 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.748100182000002, 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0058897380000000003, 2024-09-19T21:34:36.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481071+0000", 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481071+0000", 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481070+0000", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481081+0000", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0983e-05 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481116+0000", 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5463000000000002e-05 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481126+0000", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2960000000000004e-06 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.483805+0000", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026790889999999999 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.483852+0000", 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7048000000000001e-05 2024-09-19T21:34:36.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.486868+0000", 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030160320000000001 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.486961+0000", 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3023999999999996e-05 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:114 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'64)", 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.481809+0000", 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.747361972, 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0063331439999999998, 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481809+0000", 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481809+0000", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481808+0000", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481822+0000", 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4246e-05 2024-09-19T21:34:36.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481843+0000", 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1109000000000001e-05 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.481875+0000", 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1973e-05 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484881+0000", 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0030063049999999999 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484916+0000", 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4468999999999998e-05 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.488066+0000", 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031500740000000001 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.488143+0000", 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6624000000000004e-05 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:115 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'64)", 2024-09-19T21:34:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.482916+0000", 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.746254875999998, 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0052940169999999998, 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482916+0000", 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482916+0000", 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482915+0000", 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.483073+0000", 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000158763 2024-09-19T21:34:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.483095+0000", 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1403000000000002e-05 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.483204+0000", 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109195 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485743+0000", 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025394509999999999 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485805+0000", 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1192000000000006e-05 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.488149+0000", 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002344189 2024-09-19T21:34:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.488211+0000", 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1784000000000003e-05 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'87)", 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.924589+0000", 2024-09-19T21:34:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.304582739000001, 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0054295660000000003, 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924589+0000", 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924589+0000", 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924587+0000", 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924711+0000", 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000124218 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924738+0000", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6774000000000001e-05 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924752+0000", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3912e-05 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924821+0000", 2024-09-19T21:34:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8640999999999996e-05 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.924866+0000", 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5126000000000001e-05 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.929638+0000", 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047719019999999997 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.930018+0000", 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038041600000000001 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:135 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'93, mlcod=16'87)", 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.925939+0000", 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.303232613999999, 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0056635330000000001, 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925939+0000", 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925939+0000", 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925938+0000", 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925948+0000", 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8300000000000008e-06 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925969+0000", 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1812000000000001e-05 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.925978+0000", 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8170000000000005e-06 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.926081+0000", 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103299 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.926118+0000", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6888999999999999e-05 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.931484+0000", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053651879999999999 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.931602+0000", 2024-09-19T21:34:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118773 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:137 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'95, mlcod=16'88)", 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.928251+0000", 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.300920075000001, 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049924460000000002, 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928251+0000", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928251+0000", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928250+0000", 2024-09-19T21:34:36.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928299+0000", 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9156999999999997e-05 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928359+0000", 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9627999999999999e-05 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928371+0000", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1882000000000001e-05 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928484+0000", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011318400000000001 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928513+0000", 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9617000000000001e-05 2024-09-19T21:34:36.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.933177+0000", 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046633660000000004 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.933244+0000", 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6991999999999998e-05 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:138 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'96, mlcod=16'88)", 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.928986+0000", 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 19.300185210999999, 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049119710000000002, 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928986+0000", 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928986+0000", 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928985+0000", 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.928997+0000", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2252999999999999e-05 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.929014+0000", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7303e-05 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.929031+0000", 2024-09-19T21:34:36.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6540000000000001e-05 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.930090+0000", 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001059293 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.930119+0000", 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9054999999999999e-05 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.933846+0000", 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037265929999999998 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.933898+0000", 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2266999999999999e-05 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=17'105)", 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:17.591002+0000", 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.638169087000001, 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006315696, 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591002+0000", 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591002+0000", 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591001+0000", 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591025+0000", 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3992999999999999e-05 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591049+0000", 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3702000000000001e-05 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591087+0000", 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8483999999999999e-05 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591155+0000", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8374999999999995e-05 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.591192+0000", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6239000000000003e-05 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.597247+0000", 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0060549810000000001 2024-09-19T21:34:36.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.597318+0000", 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1315000000000004e-05 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=17'105)", 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:17.592650+0000", 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.636521297000002, 2024-09-19T21:34:36.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047554470000000003, 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592650+0000", 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592650+0000", 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592649+0000", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592742+0000", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3132000000000002e-05 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592836+0000", 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3820000000000006e-05 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592849+0000", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.27e-05 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.592962+0000", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011316599999999999 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.593019+0000", 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7346999999999998e-05 2024-09-19T21:34:36.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.597332+0000", 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0043128569999999998 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:17.597406+0000", 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3460999999999993e-05 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.335 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.335 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:36.335 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:36.335 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:36.336 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:36.334+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:36.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:36.437 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:36.432+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:36.447 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:36.448 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:36.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:36.539 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:36.532+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:36.566 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:36.566 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:36.566 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:36.566 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:36.566 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:36.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:36.610 INFO:teuthology.orchestra.run.smithi122.stdout:91 trunc from 0xb35fa00 to 0x5ca3000 2024-09-19T21:34:36.639 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:36.634+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:36.650 INFO:teuthology.orchestra.run.smithi122.stdout:95 read 0x20a4c00 thru 0x20aef49 (0xa34a bytes) 2024-09-19T21:34:36.657 INFO:teuthology.orchestra.run.smithi122.stdout:96 read 0x3e93600 thru 0x3e94997 (0x1398 bytes) 2024-09-19T21:34:36.691 INFO:teuthology.orchestra.run.smithi122.stdout:97 write 0xb394000 thru 0xb3986e1 (0x46e2 bytes) 2024-09-19T21:34:36.692 INFO:teuthology.orchestra.run.smithi122.stdout:98 read 0x97b9a00 thru 0x97c9014 (0xf615 bytes) 2024-09-19T21:34:36.692 INFO:teuthology.orchestra.run.smithi122.stdout:103 punch from 0x85bee00 to 0x85c4200, (0x5400 bytes) 2024-09-19T21:34:36.709 INFO:teuthology.orchestra.run.smithi122.stdout:107 trunc from 0xb398800 to 0x59c4000 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.776252266, 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0060163100000000004, 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952028+0000", 2024-09-19T21:34:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952025+0000", 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952046+0000", 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0786999999999999e-05 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952078+0000", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2021999999999999e-05 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952102+0000", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3289999999999999e-05 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952216+0000", 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000114022 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.952223+0000", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1559999999999998e-06 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955067+0000", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0028442789999999999 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955205+0000", 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138391 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.955372+0000", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000166167 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.956612+0000", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012408199999999999 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957863+0000", 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001250384 2024-09-19T21:34:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.957962+0000", 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9715000000000003e-05 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:21.958045+0000", 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2161000000000002e-05 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4224.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.773431978, 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033600710000000001, 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4224", 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.12:0/760488900", 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954849+0000", 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954846+0000", 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954861+0000", 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.452e-05 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954883+0000", 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359000000000001e-05 2024-09-19T21:34:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.954900+0000", 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7008e-05 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955053+0000", 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000153311 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955302+0000", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024876800000000001 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.955695+0000", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00039324800000000001 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.957181+0000", 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001485105 2024-09-19T21:34:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958132+0000", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095181900000000002 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958178+0000", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5419e-05 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:22.958209+0000", 2024-09-19T21:34:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0908000000000003e-05 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4358.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.008133+0000", 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.7201480369999995, 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0048982569999999996, 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008133+0000", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008133+0000", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008132+0000", 2024-09-19T21:34:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008139+0000", 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0160000000000003e-06 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008149+0000", 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7399999999999999e-06 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008157+0000", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9799999999999998e-06 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008220+0000", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3477000000000002e-05 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.008239+0000", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9157e-05 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.012989+0000", 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047495310000000004 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.013031+0000", 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1823000000000003e-05 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:13 3.3 3:c590f66e:::rbd_object_map.11068c3bf21f:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:32.022720+0000", 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.7055608959999997, 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025087629999999998, 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022720+0000", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022720+0000", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022718+0000", 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022729+0000", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0975000000000001e-05 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022754+0000", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5052999999999999e-05 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022765+0000", 2024-09-19T21:34:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1421e-05 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.022836+0000", 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0630999999999995e-05 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023012+0000", 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017607000000000001 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.023239+0000", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022694800000000001 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.024996+0000", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001757572 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.025155+0000", 2024-09-19T21:34:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015838700000000001 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.025199+0000", 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4647999999999997e-05 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:32.025228+0000", 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9186000000000001e-05 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4470.0:512 3.3 3:c590f66e:::rbd_object_map.11068c3bf21f:head [read 0~46] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.040064+0000", 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.68821666699999995, 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030977610000000001, 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4470", 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 512 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040064+0000", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040064+0000", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040062+0000", 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040073+0000", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0745e-05 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040098+0000", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5015999999999999e-05 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040113+0000", 2024-09-19T21:34:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5012e-05 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040195+0000", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1907e-05 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.040204+0000", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5550000000000008e-06 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.042997+0000", 2024-09-19T21:34:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0027929169999999998 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.043062+0000", 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4389999999999998e-05 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.043162+0000", 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010017399999999999 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:297 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.102802+0000", 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62547882899999996, 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029392530000000002, 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 297 2024-09-19T21:34:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102802+0000", 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102802+0000", 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102799+0000", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102813+0000", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.305e-05 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102839+0000", 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6499000000000002e-05 2024-09-19T21:34:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102852+0000", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3113e-05 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.102956+0000", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000104305 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.103252+0000", 2024-09-19T21:34:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000295753 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.103701+0000", 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00044869300000000001 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.105144+0000", 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014426739999999999 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.105542+0000", 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00039831600000000001 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.105659+0000", 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000117297 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.105741+0000", 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1833000000000001e-05 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:312 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.128444+0000", 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.59983686000000003, 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026921269999999999, 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 312 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128444+0000", 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128444+0000", 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128441+0000", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128456+0000", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4328000000000001e-05 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128484+0000", 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8288000000000001e-05 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128498+0000", 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4442e-05 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128596+0000", 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7431999999999998e-05 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.128834+0000", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023769399999999999 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.129134+0000", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030035700000000001 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.130470+0000", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013359299999999999 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.131008+0000", 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00053826799999999995 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.131081+0000", 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2964000000000001e-05 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.131136+0000", 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4761e-05 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4470.0:567 3.3 3:c590f66e:::rbd_object_map.11068c3bf21f:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.345253+0000", 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.38302735900000001, 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002450453, 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4470", 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1205840605", 2024-09-19T21:34:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 567 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345253+0000", 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345253+0000", 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345251+0000", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345267+0000", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6200000000000001e-05 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345303+0000", 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5865000000000003e-05 2024-09-19T21:34:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345320+0000", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6991e-05 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345426+0000", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010611 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345585+0000", 2024-09-19T21:34:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000159526 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.345928+0000", 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034241799999999999 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.347132+0000", 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012039780000000001 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.347611+0000", 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00047878599999999998 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.347663+0000", 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2707000000000002e-05 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.347704+0000", 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0343999999999999e-05 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:106 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.610694+0000", 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.11758616600000001, 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00263715, 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 106 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610694+0000", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610694+0000", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610692+0000", 2024-09-19T21:34:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610706+0000", 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.401e-05 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610736+0000", 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9363e-05 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610750+0000", 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4239e-05 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.610877+0000", 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012693399999999999 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.611143+0000", 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026552800000000001 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.611486+0000", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000343672 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.612733+0000", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012464450000000001 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.613149+0000", 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041637299999999999 2024-09-19T21:34:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.613226+0000", 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7051999999999993e-05 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.613332+0000", 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010551400000000001 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:179 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.630833+0000", 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.097447122999999997, 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029945079999999999, 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 179 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630833+0000", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630833+0000", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630832+0000", 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630840+0000", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5609999999999999e-06 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630859+0000", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8623000000000001e-05 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630867+0000", 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8150000000000006e-06 2024-09-19T21:34:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.630936+0000", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9603000000000004e-05 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631054+0000", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011788100000000001 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631525+0000", 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00047042999999999998 2024-09-19T21:34:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.632858+0000", 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001333774 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.633761+0000", 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090230699999999998 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.633791+0000", 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0766000000000003e-05 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.633828+0000", 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6551999999999998e-05 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:182 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.631364+0000", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.096916405999999997, 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002972546, 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 182 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631364+0000", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631364+0000", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631363+0000", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631370+0000", 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3799999999999999e-06 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631380+0000", 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8789999999999998e-06 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.631388+0000", 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8419999999999994e-06 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.632055+0000", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000666108 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.632181+0000", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012681899999999999 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.632606+0000", 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00042437600000000002 2024-09-19T21:34:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.633409+0000", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000802862 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.634289+0000", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00088009700000000004 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.634316+0000", 2024-09-19T21:34:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7521999999999998e-05 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.634337+0000", 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:211 3.c 3:306a6325:::rbd_data.11068c3bf21f.0000000000000008:4 [read 712704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.650641+0000", 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.077639641999999995, 2024-09-19T21:34:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026704749999999998, 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 211 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650641+0000", 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650641+0000", 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650641+0000", 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650642+0000", 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6190000000000001e-06 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650646+0000", 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6770000000000001e-06 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.650649+0000", 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4699999999999998e-06 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.653025+0000", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0023752560000000001 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.653160+0000", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000135532 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.653311+0000", 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015120699999999999 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:239 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.674383+0000", 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.053897619000000001, 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029038330000000002, 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 239 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674383+0000", 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674383+0000", 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674380+0000", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674394+0000", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3957e-05 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674423+0000", 2024-09-19T21:34:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9005000000000001e-05 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674437+0000", 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3244999999999999e-05 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674537+0000", 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9895999999999994e-05 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.674679+0000", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000142898 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.675398+0000", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000718985 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.676686+0000", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001287285 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.677187+0000", 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00050154099999999999 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.677247+0000", 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0038999999999999e-05 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.677287+0000", 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9465000000000002e-05 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:265 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.689965+0000", 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.038315389999999998, 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.005562244, 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 265 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.689965+0000", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.689965+0000", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.689963+0000", 2024-09-19T21:34:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.689973+0000", 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0251000000000001e-05 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.689994+0000", 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1024999999999999e-05 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690004+0000", 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2259999999999994e-06 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690094+0000", 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0340000000000006e-05 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690232+0000", 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013769100000000001 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690751+0000", 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000519569 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695262+0000", 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0045110630000000001 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695391+0000", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000128532 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695472+0000", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1403999999999995e-05 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695527+0000", 2024-09-19T21:34:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.516e-05 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:266 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.690018+0000", 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.038262097000000002, 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0059974240000000003, 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 266 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690018+0000", 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690018+0000", 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690018+0000", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690022+0000", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4359999999999998e-06 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690027+0000", 2024-09-19T21:34:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0150000000000003e-06 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690030+0000", 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8870000000000002e-06 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.691526+0000", 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014956749999999999 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.691625+0000", 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8771999999999996e-05 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.691971+0000", 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034603500000000002 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695590+0000", 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036195440000000001 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695850+0000", 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259478 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695944+0000", 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4040000000000001e-05 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696016+0000", 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1962e-05 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:267 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.690921+0000", 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.037359445999999998, 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052070479999999997, 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 267 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690921+0000", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690921+0000", 2024-09-19T21:34:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690920+0000", 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690927+0000", 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0469999999999996e-06 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690940+0000", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2411e-05 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690948+0000", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2959999999999999e-06 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692520+0000", 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0015718570000000001 2024-09-19T21:34:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692824+0000", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030458800000000001 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.693367+0000", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00054223400000000005 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695814+0000", 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024475909999999998 2024-09-19T21:34:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696029+0000", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021448499999999999 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696069+0000", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0652999999999999e-05 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696128+0000", 2024-09-19T21:34:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8946000000000001e-05 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:268 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.690955+0000", 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.037325656999999998, 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0055564289999999999, 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 268 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690955+0000", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690955+0000", 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690955+0000", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690958+0000", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.18e-06 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690964+0000", 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9769999999999997e-06 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.690968+0000", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.693996+0000", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030287030000000002 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.694113+0000", 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116235 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.694738+0000", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062494199999999997 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696139+0000", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001401326 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696423+0000", 2024-09-19T21:34:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028433799999999997 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696471+0000", 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8065000000000001e-05 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.696511+0000", 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9996999999999997e-05 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop_reply(client.4512.0:265 3.f e28/23 ondisk, result = 0)", 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.692815+0000", 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.035465889, 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0027282510000000001, 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692815+0000", 2024-09-19T21:34:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692815+0000", 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692813+0000", 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692825+0000", 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1236999999999999e-05 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692844+0000", 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9806999999999999e-05 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.692855+0000", 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0907000000000001e-05 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695369+0000", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002513277 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695382+0000", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3832000000000001e-05 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.695543+0000", 2024-09-19T21:34:36.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016047899999999999 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:285 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.712238+0000", 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.016042858, 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030312820000000002, 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 285 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712238+0000", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712238+0000", 2024-09-19T21:34:36.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712236+0000", 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712248+0000", 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.224e-05 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712270+0000", 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1559000000000001e-05 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712280+0000", 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0631e-05 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712358+0000", 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7844999999999998e-05 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.712539+0000", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018071799999999999 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.713270+0000", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00073156499999999999 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.714716+0000", 2024-09-19T21:34:36.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014459010000000001 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.715132+0000", 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000415468 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.715189+0000", 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7457000000000002e-05 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.715269+0000", 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9745000000000003e-05 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:313 3.f 3:f67ac732:::rbd_object_map.1106b3d15a22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.724979+0000", 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0033019640000000001, 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026439760000000001, 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.844 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 313 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.724979+0000", 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.724979+0000", 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.724977+0000", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.724986+0000", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6340000000000007e-06 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.725003+0000", 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7813999999999999e-05 2024-09-19T21:34:36.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.725015+0000", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1685000000000001e-05 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.725301+0000", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028599400000000003 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.725450+0000", 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014835299999999999 2024-09-19T21:34:36.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.726026+0000", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057610099999999998 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.726866+0000", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00083989000000000004 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.727511+0000", 2024-09-19T21:34:36.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00064513799999999996 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.727565+0000", 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4703999999999997e-05 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.727623+0000", 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7253000000000002e-05 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.849 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.850 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.850 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:36.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:36.852 INFO:teuthology.orchestra.run.smithi122.stdout:111 punch from 0x1cdbc00 to 0x1cdf600, (0x3a00 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:115 punch from 0x11fee00 to 0x1200e00, (0x2000 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:116 write 0xec30000 thru 0xec376e1 (0x76e2 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:117 read 0xc848e00 thru 0xc84ac92 (0x1e93 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:118 write 0x8807200 thru 0x8810504 (0x9305 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:120 punch from 0x2cb4800 to 0x2cb4e00, (0x600 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:122 write 0xb535200 thru 0xb54052b (0xb32c bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:123 read 0x2617400 thru 0x26256ef (0xe2f0 bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:127 write 0x9d3b800 thru 0x9d44e0e (0x960f bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:128 read 0x3c3600 thru 0x3d22ec (0xeced bytes) 2024-09-19T21:34:36.853 INFO:teuthology.orchestra.run.smithi122.stdout:129 trunc from 0xec37800 to 0x7750c00 2024-09-19T21:34:36.854 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:36.846+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:36.864 INFO:teuthology.orchestra.run.smithi122.stdout:130 read 0x56f5c00 thru 0x56fb435 (0x5836 bytes) 2024-09-19T21:34:36.866 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xb8b1e00 2024-09-19T21:34:36.876 INFO:teuthology.orchestra.run.smithi122.stdout:131 trunc from 0x7750c00 to 0xb8b1e00 2024-09-19T21:34:36.894 INFO:teuthology.orchestra.run.smithi122.stdout:132 write 0x86a4400 thru 0x86a8a37 (0x4638 bytes) 2024-09-19T21:34:36.896 INFO:teuthology.orchestra.run.smithi122.stdout:133 punch from 0x3d97e00 to 0x3da1000, (0x9200 bytes) 2024-09-19T21:34:36.898 INFO:teuthology.orchestra.run.smithi122.stdout:135 punch from 0x1b1c000 to 0x1b1f000, (0x3000 bytes) 2024-09-19T21:34:36.900 INFO:teuthology.orchestra.run.smithi122.stdout:136 read 0xb84d400 thru 0xb856a37 (0x9638 bytes) 2024-09-19T21:34:36.900 INFO:teuthology.orchestra.run.smithi122.stdout:137 write 0x2e65000 thru 0x2e69349 (0x434a bytes) 2024-09-19T21:34:36.901 INFO:teuthology.orchestra.run.smithi122.stdout:139 punch from 0x8882600 to 0x8892400, (0xfe00 bytes) 2024-09-19T21:34:36.903 INFO:teuthology.orchestra.run.smithi122.stdout:144 read 0x76eae00 thru 0x76ed6c7 (0x28c8 bytes) 2024-09-19T21:34:36.943 INFO:teuthology.orchestra.run.smithi122.stdout:147 trunc from 0xb8b1e00 to 0x188b200 2024-09-19T21:34:36.952 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:36.950+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:581 3.1c 3.c02ad67c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.974668+0000", 2024-09-19T21:34:36.978 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.3411000000000002e-05, 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7991000000000002e-05, 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 581 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974668+0000", 2024-09-19T21:34:36.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974668+0000", 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974667+0000", 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974674+0000", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8560000000000001e-06 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974689+0000", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.556e-05 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974697+0000", 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7649999999999993e-06 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.981 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:478 3.1c 3.c02ad67c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.974698+0000", 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.6490000000000001e-06, 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1467e-05, 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 478 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:36.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974698+0000", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974698+0000", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974696+0000", 2024-09-19T21:34:36.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974702+0000", 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9789999999999998e-06 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974715+0000", 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2316e-05 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:36.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.974720+0000", 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3070000000000002e-06 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 2 2024-09-19T21:34:36.985 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:36.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:37.019 INFO:teuthology.orchestra.run.smithi122.stdout:148 write 0xd375c00 thru 0xd376026 (0x427 bytes) 2024-09-19T21:34:37.021 INFO:teuthology.orchestra.run.smithi122.stdout:149 punch from 0x1323c00 to 0x1325c00, (0x2000 bytes) 2024-09-19T21:34:37.022 INFO:teuthology.orchestra.run.smithi122.stdout:150 read 0x948ec00 thru 0x9498beb (0x9fec bytes) 2024-09-19T21:34:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-09-19T21:34:37.053 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:37.044+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:37.059 INFO:teuthology.orchestra.run.smithi122.stdout:153 write 0xd393c00 thru 0xd39b453 (0x7854 bytes) 2024-09-19T21:34:37.061 INFO:teuthology.orchestra.run.smithi122.stdout:156 write 0x8c2fa00 thru 0x8c356ac (0x5cad bytes) 2024-09-19T21:34:37.061 INFO:teuthology.orchestra.run.smithi122.stdout:162 read 0x910ae00 thru 0x9115c8f (0xae90 bytes) 2024-09-19T21:34:37.062 INFO:teuthology.orchestra.run.smithi122.stdout:163 read 0xc448e00 thru 0xc44f624 (0x6825 bytes) 2024-09-19T21:34:37.063 INFO:teuthology.orchestra.run.smithi122.stdout:166 read 0xa736400 thru 0xa745cc5 (0xf8c6 bytes) 2024-09-19T21:34:37.063 INFO:teuthology.orchestra.run.smithi122.stdout:167 punch from 0x4a58c00 to 0x4a65800, (0xcc00 bytes) 2024-09-19T21:34:37.063 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-19T21:34:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:168 read 0x8e7b800 thru 0x8e8b5bf (0xfdc0 bytes) 2024-09-19T21:34:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:170 read 0x8419200 thru 0x841f8c7 (0x66c8 bytes) 2024-09-19T21:34:37.066 INFO:teuthology.orchestra.run.smithi122.stdout:171 punch from 0xa86e00 to 0xa8fc00, (0x8e00 bytes) 2024-09-19T21:34:37.068 INFO:teuthology.orchestra.run.smithi122.stdout:176 read 0x3131600 thru 0x31354a7 (0x3ea8 bytes) 2024-09-19T21:34:37.068 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xbe7d800 2024-09-19T21:34:37.071 INFO:teuthology.orchestra.run.smithi122.stdout:177 trunc from 0xd39b600 to 0xbe7d800 2024-09-19T21:34:37.103 INFO:teuthology.orchestra.run.smithi122.stdout:178 read 0x1a47a00 thru 0x1a47cbf (0x2c0 bytes) 2024-09-19T21:34:37.104 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:37.105 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:37.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:37.125 INFO:teuthology.orchestra.run.smithi122.stdout:182 trunc from 0xbe7d800 to 0x270f600 2024-09-19T21:34:37.154 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:37.150+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:37.157 INFO:teuthology.orchestra.run.smithi122.stdout:183 read 0x24ba400 thru 0x24c825e (0xde5f bytes) 2024-09-19T21:34:37.158 INFO:teuthology.orchestra.run.smithi122.stdout:190 punch from 0x9f5800 to 0x9ff400, (0x9c00 bytes) 2024-09-19T21:34:37.160 INFO:teuthology.orchestra.run.smithi122.stdout:192 read 0x1f4b400 thru 0x1f4dccf (0x28d0 bytes) 2024-09-19T21:34:37.181 INFO:teuthology.orchestra.run.smithi122.stdout:193 write 0x4d12600 thru 0x4d1e045 (0xba46 bytes) 2024-09-19T21:34:37.190 INFO:teuthology.orchestra.run.smithi122.stdout:196 trunc from 0x4d1e200 to 0x22f0200 2024-09-19T21:34:37.234 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:37.234 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:37.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.29520698, 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:34:37.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:34:37.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:34:37.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.052288938, 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011699008, 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 37 2024-09-19T21:34:37.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175717+0000", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175721+0000", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5700000000000003e-06 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175741+0000", 2024-09-19T21:34:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0341999999999999e-05 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.175748+0000", 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4230000000000002e-06 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185384+0000", 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0096361899999999993 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185499+0000", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000114843 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.185626+0000", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012677599999999999 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.186404+0000", 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000778658 2024-09-19T21:34:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187288+0000", 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000883179 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187363+0000", 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5041000000000002e-05 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.187416+0000", 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.965119220999998, 2024-09-19T21:34:37.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012081382, 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 87 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:37.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262887+0000", 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262886+0000", 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.262994+0000", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108334 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263011+0000", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6419999999999999e-05 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263023+0000", 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2663000000000001e-05 2024-09-19T21:34:37.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268657+0000", 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056333720000000002 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268794+0000", 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000137331 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.268985+0000", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019139100000000001 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.271593+0000", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002608057 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274842+0000", 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0032483780000000001 2024-09-19T21:34:37.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274910+0000", 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8592999999999999e-05 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.274968+0000", 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7710000000000001e-05 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.964968311, 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012663806999999999, 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 88 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263038+0000", 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263037+0000", 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263047+0000", 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0172e-05 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263101+0000", 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4493000000000002e-05 2024-09-19T21:34:37.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.263115+0000", 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3733e-05 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270291+0000", 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0071757640000000003 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270447+0000", 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000156291 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.270753+0000", 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030557000000000002 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.271983+0000", 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001230243 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275563+0000", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035797340000000002 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275642+0000", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9558e-05 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.275701+0000", 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9138999999999997e-05 2024-09-19T21:34:37.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.755255584, 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.01141986, 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 112 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472750+0000", 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472889+0000", 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000139197 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472909+0000", 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9649000000000001e-05 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472917+0000", 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7439999999999996e-06 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477287+0000", 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0043697110000000001 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477358+0000", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0549000000000005e-05 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.477484+0000", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000126119 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482197+0000", 2024-09-19T21:34:37.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047129190000000003 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484110+0000", 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001913503 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484142+0000", 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2072000000000001e-05 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484170+0000", 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8073999999999999e-05 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.755079384999998, 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011770846999999999, 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 113 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472927+0000", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472926+0000", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472934+0000", 2024-09-19T21:34:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6570000000000003e-06 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472946+0000", 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1365999999999999e-05 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472953+0000", 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0299999999999996e-06 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478126+0000", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051735970000000003 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478206+0000", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9552e-05 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478320+0000", 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011373400000000001 2024-09-19T21:34:37.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482477+0000", 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0041574489999999997 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484576+0000", 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0020983210000000002 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484626+0000", 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0322999999999999e-05 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484697+0000", 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1607999999999996e-05 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.273 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.755041316, 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012975662000000001, 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 114 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472965+0000", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472964+0000", 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472970+0000", 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4369999999999998e-06 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472979+0000", 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8800000000000003e-06 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.472986+0000", 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9859999999999997e-06 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478857+0000", 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0058707689999999996 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.478929+0000", 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1633999999999999e-05 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479048+0000", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119163 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482618+0000", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003570045 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485769+0000", 2024-09-19T21:34:37.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031512670000000001 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485865+0000", 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6063999999999994e-05 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.485940+0000", 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5030000000000005e-05 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.754640185, 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012803063, 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 115 2024-09-19T21:34:37.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473366+0000", 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473364+0000", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473518+0000", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000153263 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473549+0000", 2024-09-19T21:34:37.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0945999999999998e-05 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473559+0000", 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0246e-05 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479723+0000", 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0061640319999999998 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.479860+0000", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013705100000000001 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.480132+0000", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027181599999999997 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482903+0000", 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002771187 2024-09-19T21:34:37.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486041+0000", 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003138278 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486115+0000", 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3968999999999996e-05 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486169+0000", 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3767999999999997e-05 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.754429536, 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012816342999999999, 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.286 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 116 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473576+0000", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473581+0000", 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8329999999999999e-06 2024-09-19T21:34:37.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473612+0000", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0009e-05 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473620+0000", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5469999999999997e-06 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481072+0000", 2024-09-19T21:34:37.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0074516890000000001 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481221+0000", 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149073 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.481419+0000", 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000198407 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.482910+0000", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014911460000000001 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486256+0000", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00334557 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486309+0000", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3047000000000002e-05 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.486393+0000", 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3821999999999998e-05 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.754371194000001, 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014549436000000001, 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 117 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473635+0000", 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473634+0000", 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473640+0000", 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6699999999999999e-06 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473650+0000", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0266e-05 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.473662+0000", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.146e-05 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483033+0000", 2024-09-19T21:34:37.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00937114 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483198+0000", 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016553 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.483387+0000", 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018844000000000001 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.484203+0000", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00081628699999999996 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488045+0000", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0038421129999999999 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488104+0000", 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8486000000000002e-05 2024-09-19T21:34:37.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.488184+0000", 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0704000000000005e-05 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.313111369000001, 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014174482, 2024-09-19T21:34:37.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 132 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914895+0000", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914894+0000", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914901+0000", 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5900000000000002e-06 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914911+0000", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7229999999999999e-06 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.914918+0000", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8360000000000003e-06 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918602+0000", 2024-09-19T21:34:37.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003684091 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918752+0000", 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149796 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.918958+0000", 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020586899999999999 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923728+0000", 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047697879999999996 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928934+0000", 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052066279999999996 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929014+0000", 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9725e-05 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929069+0000", 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5245999999999999e-05 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312894453999998, 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014071554, 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 133 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915112+0000", 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915110+0000", 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915118+0000", 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8299999999999996e-06 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915132+0000", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4103e-05 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915145+0000", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2096000000000001e-05 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920244+0000", 2024-09-19T21:34:37.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050993480000000001 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920394+0000", 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000150517 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.920552+0000", 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015807700000000001 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923918+0000", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033651670000000001 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929084+0000", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051667390000000001 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929141+0000", 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6990999999999997e-05 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929183+0000", 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1736000000000002e-05 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312839504999999, 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015859873, 2024-09-19T21:34:37.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 134 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915166+0000", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915336+0000", 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000170736 2024-09-19T21:34:37.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915372+0000", 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5828999999999999e-05 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915382+0000", 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8300000000000008e-06 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921117+0000", 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057345859999999999 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921363+0000", 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024658200000000002 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.921710+0000", 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034673199999999999 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928915+0000", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0072048800000000003 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.930772+0000", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001857214 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.930906+0000", 2024-09-19T21:34:37.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013437 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931026+0000", 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012006399999999999 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.316 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312611617000002, 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015795664000000001, 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 135 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915394+0000", 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915401+0000", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0600000000000002e-06 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915410+0000", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3670000000000003e-06 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915420+0000", 2024-09-19T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0396e-05 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923006+0000", 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0075852269999999999 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923081+0000", 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5186999999999997e-05 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923194+0000", 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113071 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929854+0000", 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0066603610000000001 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931048+0000", 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011941090000000001 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931125+0000", 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6553999999999999e-05 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931190+0000", 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5129000000000006e-05 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312578418000001, 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015919566, 2024-09-19T21:34:37.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 136 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915428+0000", 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915427+0000", 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915433+0000", 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.993e-06 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915441+0000", 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915458+0000", 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6820000000000002e-05 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.923948+0000", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0084897940000000002 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.924079+0000", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130877 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.924265+0000", 2024-09-19T21:34:37.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185899 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929920+0000", 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056554149999999996 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931211+0000", 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012910140000000001 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931284+0000", 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3077999999999994e-05 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931347+0000", 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2996000000000005e-05 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312537381999999, 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016022577999999999, 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 137 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915469+0000", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915468+0000", 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915623+0000", 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015469600000000001 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915640+0000", 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7387000000000001e-05 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915652+0000", 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2112e-05 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925179+0000", 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0095269759999999995 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925263+0000", 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3497000000000004e-05 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.925373+0000", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110048 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931369+0000", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0059966840000000004 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931425+0000", 2024-09-19T21:34:37.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5809e-05 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931467+0000", 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2107999999999997e-05 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931491+0000", 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3924e-05 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312344032999999, 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015929169999999999, 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 138 2024-09-19T21:34:37.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915662+0000", 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915661+0000", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915668+0000", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915692+0000", 2024-09-19T21:34:37.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4352999999999999e-05 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915701+0000", 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6270000000000006e-06 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926019+0000", 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010317941000000001 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926090+0000", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1846000000000003e-05 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926193+0000", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102983 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931384+0000", 2024-09-19T21:34:37.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051904070000000002 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931533+0000", 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014889499999999999 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931569+0000", 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6578000000000001e-05 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931591+0000", 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1863999999999999e-05 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312231079, 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017414302, 2024-09-19T21:34:37.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 139 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915775+0000", 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915774+0000", 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915783+0000", 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7129999999999994e-06 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915792+0000", 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2129999999999997e-06 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915803+0000", 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0793e-05 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926692+0000", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010888616 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.926828+0000", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013679100000000001 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.927017+0000", 2024-09-19T21:34:37.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000188729 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931628+0000", 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0046104650000000002 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933068+0000", 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001440167 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933149+0000", 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1642000000000002e-05 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933189+0000", 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9805999999999998e-05 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312192205999999, 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017585809000000001, 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 140 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.349 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915814+0000", 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915813+0000", 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915954+0000", 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014095000000000001 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915964+0000", 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.039e-05 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915972+0000", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928045+0000", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012073599000000001 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928122+0000", 2024-09-19T21:34:37.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6210000000000004e-05 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.928231+0000", 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109438 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.931841+0000", 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036096019999999999 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933200+0000", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013596280000000001 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933237+0000", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6671000000000002e-05 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933400+0000", 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000162652 2024-09-19T21:34:37.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.312025847000001, 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017506562, 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:37.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 141 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915980+0000", 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915979+0000", 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915989+0000", 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.915997+0000", 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2360000000000004e-06 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.916004+0000", 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6930000000000001e-06 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929219+0000", 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013214755 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929299+0000", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0203000000000004e-05 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.929408+0000", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108924 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.932599+0000", 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003191138 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933417+0000", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00081806700000000001 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933461+0000", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4462999999999998e-05 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.933487+0000", 2024-09-19T21:34:37.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5216e-05 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.361 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:37.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:37.358+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:37.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:200 write 0xdc30800 thru 0xdc38a0d (0x820e bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:203 punch from 0x1f04a00 to 0x1f0b000, (0x6600 bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:204 punch from 0x62b5200 to 0x62c3600, (0xe400 bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:205 read 0x1ac6600 thru 0x1acb2dd (0x4cde bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:206 write 0x2921600 thru 0x2927631 (0x6032 bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:207 write 0x6aeb600 thru 0x6aef019 (0x3a1a bytes) 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:209 trunc from 0xdc38c00 to 0x3b68e00 2024-09-19T21:34:37.364 INFO:teuthology.orchestra.run.smithi122.stdout:212 flatten 2024-09-19T21:34:37.463 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:37.458+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:37.472 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:37.472 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:37.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:37.564 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:37.562+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:37.566 INFO:teuthology.orchestra.run.smithi122.stdout:216 read 0x20e4800 thru 0x20ea4a8 (0x5ca9 bytes) 2024-09-19T21:34:37.567 INFO:teuthology.orchestra.run.smithi122.stdout:221 punch from 0xd4400 to 0xda600, (0x6200 bytes) 2024-09-19T21:34:37.592 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:37.592 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:37.592 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:37.592 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:37.592 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:37.593 INFO:teuthology.orchestra.run.smithi122.stdout:222 write 0x7723200 thru 0x7731efe (0xecff bytes) 2024-09-19T21:34:37.594 INFO:teuthology.orchestra.run.smithi122.stdout:224 punch from 0x462f000 to 0x4637c00, (0x8c00 bytes) 2024-09-19T21:34:37.595 INFO:teuthology.orchestra.run.smithi122.stdout:225 read 0x83c000 thru 0x84bbed (0xfbee bytes) 2024-09-19T21:34:37.596 INFO:teuthology.orchestra.run.smithi122.stdout:227 punch from 0x66e5600 to 0x66ea600, (0x5000 bytes) 2024-09-19T21:34:37.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:37.625 INFO:teuthology.orchestra.run.smithi122.stdout:228 write 0xd347200 thru 0xd351fd9 (0xadda bytes) 2024-09-19T21:34:37.625 INFO:teuthology.orchestra.run.smithi122.stdout:230 punch from 0x5d4b400 to 0x5d4b800, (0x400 bytes) 2024-09-19T21:34:37.626 INFO:teuthology.orchestra.run.smithi122.stdout:232 write 0x3e55e00 thru 0x3e59228 (0x3429 bytes) 2024-09-19T21:34:37.627 INFO:teuthology.orchestra.run.smithi122.stdout:234 read 0x5d0dc00 thru 0x5d0f546 (0x1947 bytes) 2024-09-19T21:34:37.627 INFO:teuthology.orchestra.run.smithi122.stdout:238 read 0xa95400 thru 0xaa4e24 (0xfa25 bytes) 2024-09-19T21:34:37.629 INFO:teuthology.orchestra.run.smithi122.stdout:239 write 0x26c0a00 thru 0x26c7432 (0x6a33 bytes) 2024-09-19T21:34:37.629 INFO:teuthology.orchestra.run.smithi122.stdout:240 read 0xc79200 thru 0xc842ed (0xb0ee bytes) 2024-09-19T21:34:37.630 INFO:teuthology.orchestra.run.smithi122.stdout:241 punch from 0xb697200 to 0xb698600, (0x1400 bytes) 2024-09-19T21:34:37.631 INFO:teuthology.orchestra.run.smithi122.stdout:244 read 0xc256200 thru 0xc259b32 (0x3933 bytes) 2024-09-19T21:34:37.631 INFO:teuthology.orchestra.run.smithi122.stdout:245 read 0xbb4b600 thru 0xbb54bd7 (0x95d8 bytes) 2024-09-19T21:34:37.645 INFO:teuthology.orchestra.run.smithi122.stdout:247 trunc from 0xd352000 to 0x61c7200 2024-09-19T21:34:37.665 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:37.656+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:37.673 INFO:teuthology.orchestra.run.smithi122.stdout:249 read 0x3e24200 thru 0x3e28767 (0x4568 bytes) 2024-09-19T21:34:37.701 INFO:teuthology.orchestra.run.smithi122.stdout:250 write 0xa657400 thru 0xa65e7e1 (0x73e2 bytes) 2024-09-19T21:34:37.704 INFO:teuthology.orchestra.run.smithi122.stdout:251 write 0x2e3b800 thru 0x2e4a632 (0xee33 bytes) 2024-09-19T21:34:37.711 INFO:teuthology.orchestra.run.smithi122.stdout:252 trunc from 0xa65e800 to 0x7603600 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4224.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.762702265000001, 2024-09-19T21:34:37.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0054848090000000002, 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955033+0000", 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955032+0000", 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955040+0000", 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8399999999999995e-06 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955053+0000", 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2928000000000001e-05 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955067+0000", 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3743999999999999e-05 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955149+0000", 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2071999999999996e-05 2024-09-19T21:34:37.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.955180+0000", 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0929000000000002e-05 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.960458+0000", 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052780129999999998 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:20.960517+0000", 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9728000000000001e-05 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4512.0:298 3.2 e28/23 3:45d0fe23:::rbd_data.1106b3d15a22.0000000000000166:head v 28'8, mlcod=28'6)", 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.716799+0000", 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.000936, 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034585340000000001, 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716799+0000", 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716799+0000", 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716798+0000", 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716808+0000", 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0397999999999999e-05 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716827+0000", 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8459999999999999e-05 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716838+0000", 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0978999999999999e-05 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.716946+0000", 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108294 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.717050+0000", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010455600000000001 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.720212+0000", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031616069999999999 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.720257+0000", 2024-09-19T21:34:37.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5422e-05 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:349 3.5 3:a1bf64ae:::rbd_data.1106b3d15a22.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.752253+0000", 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.96548238200000003, 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002981226, 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 349 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752253+0000", 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752253+0000", 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752251+0000", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752261+0000", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4280000000000002e-06 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752274+0000", 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3519999999999999e-05 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752284+0000", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9669999999999996e-06 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752353+0000", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8595000000000003e-05 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752509+0000", 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015636400000000001 2024-09-19T21:34:37.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 3", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.752738+0000", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022857300000000001 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.754889+0000", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002151371 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.755162+0000", 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000272551 2024-09-19T21:34:37.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.755207+0000", 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4926999999999999e-05 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.755234+0000", 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7120000000000001e-05 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:505 3.e 3:753a3fdb:::rbd_data.1106b3d15a22.000000000000010d:head [set-alloc-hint object_size 524288 write_size 524288,write 335872~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.901769+0000", 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.81596576300000001, 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0041308600000000001, 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 505 2024-09-19T21:34:37.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901769+0000", 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901769+0000", 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901766+0000", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901795+0000", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8328000000000001e-05 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901831+0000", 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.612e-05 2024-09-19T21:34:37.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901848+0000", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7017e-05 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.901928+0000", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0118999999999996e-05 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902089+0000", 2024-09-19T21:34:37.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000160729 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902247+0000", 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015784800000000001 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.903290+0000", 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001043654 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905639+0000", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0023490249999999998 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905851+0000", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021191499999999999 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905900+0000", 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8863000000000003e-05 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.745 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:507 3.1d 3:be0eb54d:::rbd_data.1106b3d15a22.000000000000007b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 310784~16896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.902143+0000", 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.81559145, 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0037020809999999999, 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 507 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902143+0000", 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902143+0000", 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902142+0000", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902182+0000", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9759000000000003e-05 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902197+0000", 2024-09-19T21:34:37.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4895999999999999e-05 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902206+0000", 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2620000000000004e-06 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902296+0000", 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9678000000000004e-05 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902445+0000", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149145 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.902732+0000", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000287614 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.904534+0000", 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001801616 2024-09-19T21:34:37.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905726+0000", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011916120000000001 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905798+0000", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2144000000000006e-05 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.905846+0000", 2024-09-19T21:34:37.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7738999999999998e-05 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:523 3.e 3:754bab98:::rbd_data.1106b3d15a22.000000000000005c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.948156+0000", 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76957846299999999, 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035679240000000001, 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 523 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948156+0000", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948156+0000", 2024-09-19T21:34:37.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948154+0000", 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948174+0000", 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9822000000000001e-05 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948223+0000", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9774000000000002e-05 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948244+0000", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0924000000000002e-05 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948377+0000", 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013274900000000001 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948641+0000", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026364699999999999 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948860+0000", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021903499999999999 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:37.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0022408340000000001 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951611+0000", 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00051091099999999996 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951673+0000", 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1094999999999997e-05 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951724+0000", 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1878e-05 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:528 3.1d 3:be0eb54d:::rbd_data.1106b3d15a22.000000000000007b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.948328+0000", 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76940661499999996, 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0040050930000000004, 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 528 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948328+0000", 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948328+0000", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948327+0000", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948347+0000", 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0472e-05 2024-09-19T21:34:37.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948387+0000", 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9944000000000002e-05 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948418+0000", 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0467000000000002e-05 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948510+0000", 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1774000000000006e-05 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948670+0000", 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016070200000000001 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948949+0000", 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027877300000000001 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951024+0000", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0020751089999999999 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952192+0000", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011679609999999999 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952270+0000", 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8415000000000001e-05 2024-09-19T21:34:37.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952333+0000", 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2917999999999996e-05 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:530 3.11 3:8a3c54b4:::rbd_data.1106b3d15a22.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.948452+0000", 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76928260199999998, 2024-09-19T21:34:37.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0027359379999999998, 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 530 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948452+0000", 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948452+0000", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948451+0000", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948459+0000", 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.255e-06 2024-09-19T21:34:37.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948483+0000", 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4592000000000001e-05 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948495+0000", 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1469e-05 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948579+0000", 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4110000000000006e-05 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948662+0000", 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3523999999999995e-05 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.948781+0000", 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119157 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.950477+0000", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016954540000000001 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951032+0000", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00055515600000000001 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951123+0000", 2024-09-19T21:34:37.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0906e-05 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951188+0000", 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5398e-05 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:537 3.1d 3:bd868787:::rbd_data.1106b3d15a22.00000000000000b3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.951462+0000", 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.766272446, 2024-09-19T21:34:37.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0054202749999999996, 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 537 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951462+0000", 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951462+0000", 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951461+0000", 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951471+0000", 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.039e-05 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951496+0000", 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5168999999999999e-05 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951511+0000", 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4738e-05 2024-09-19T21:34:37.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951605+0000", 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4137999999999999e-05 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951698+0000", 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3064000000000002e-05 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951817+0000", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000118585 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.954148+0000", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0023312799999999998 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956821+0000", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026727589999999998 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956856+0000", 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5447999999999997e-05 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956883+0000", 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6454000000000001e-05 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:542 3.e 3:753a3fdb:::rbd_data.1106b3d15a22.000000000000010d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.951979+0000", 2024-09-19T21:34:37.776 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76575576899999998, 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057221529999999998, 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 542 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951979+0000", 2024-09-19T21:34:37.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951979+0000", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951978+0000", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951985+0000", 2024-09-19T21:34:37.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5170000000000002e-06 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.951998+0000", 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2819e-05 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952008+0000", 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3209999999999994e-06 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952065+0000", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7080999999999998e-05 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952159+0000", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4679999999999995e-05 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.952289+0000", 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012930099999999999 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956821+0000", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0045324190000000002 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.957594+0000", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000773122 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.957657+0000", 2024-09-19T21:34:37.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2312999999999998e-05 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.957701+0000", 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4811999999999997e-05 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4512.0:545 3.1 e28/23 3:828565c0:::rbd_data.1106b3d15a22.0000000000000110:head v 28'53, mlcod=28'51)", 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:36.953102+0000", 2024-09-19T21:34:37.782 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76463335700000001, 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003665626, 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953102+0000", 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953102+0000", 2024-09-19T21:34:37.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953101+0000", 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953109+0000", 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0560000000000005e-06 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953125+0000", 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6339000000000001e-05 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953137+0000", 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1671999999999999e-05 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953228+0000", 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1734e-05 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.953260+0000", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1973e-05 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956721+0000", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034604179999999998 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:36.956767+0000", 2024-09-19T21:34:37.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6474000000000003e-05 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4512.0:622 3.8 e28/23 3:15457f6e:::rbd_data.1106b3d15a22.0000000000000026:head v 28'2, mlcod=0'0)", 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.028068+0000", 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.68966725600000001, 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0028277910000000001, 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028068+0000", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028068+0000", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028066+0000", 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028087+0000", 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1072000000000001e-05 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028118+0000", 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0750999999999997e-05 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028132+0000", 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4282e-05 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028193+0000", 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1122000000000002e-05 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.028216+0000", 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2266999999999999e-05 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.030865+0000", 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002648919 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.030895+0000", 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0888e-05 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.791 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4512.0:625 3.0 e28/23 3:06365caa:::rbd_data.1106b3d15a22.0000000000000027:head v 28'4, mlcod=28'2)", 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.029827+0000", 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.687908356, 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0037315009999999999, 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029827+0000", 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029827+0000", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029826+0000", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029841+0000", 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5862000000000002e-05 2024-09-19T21:34:37.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029859+0000", 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7787000000000001e-05 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029867+0000", 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5780000000000001e-06 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029908+0000", 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0883000000000001e-05 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.029926+0000", 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7951000000000001e-05 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.033510+0000", 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035844230000000002 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.033558+0000", 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8071000000000001e-05 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:858 3.e 3:75dada11:::rbd_data.1106b3d15a22.00000000000001b8:head [set-alloc-hint object_size 524288 write_size 524288,write 100352~17408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.264699+0000", 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.45303618099999998, 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030727290000000002, 2024-09-19T21:34:37.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 858 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264699+0000", 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264699+0000", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264697+0000", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264724+0000", 2024-09-19T21:34:37.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6879000000000001e-05 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264743+0000", 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9494e-05 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264756+0000", 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3326000000000001e-05 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264822+0000", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5412000000000003e-05 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.264993+0000", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017146800000000001 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.265172+0000", 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000178333 2024-09-19T21:34:37.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.266196+0000", 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010247380000000001 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.267663+0000", 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001466937 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.267723+0000", 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9456000000000001e-05 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.267771+0000", 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8786999999999999e-05 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:900 3.e 3:75dada11:::rbd_data.1106b3d15a22.00000000000001b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.299877+0000", 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.41785831699999998, 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0027705830000000001, 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 900 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299877+0000", 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299877+0000", 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299875+0000", 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299894+0000", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9224000000000001e-05 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299914+0000", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9868999999999999e-05 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.299928+0000", 2024-09-19T21:34:37.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4098e-05 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.300001+0000", 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3077000000000005e-05 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.300150+0000", 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000148212 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.300269+0000", 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119505 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.301303+0000", 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001033759 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.302518+0000", 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012150559999999999 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.302577+0000", 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8575000000000001e-05 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.302647+0000", 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.808 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:2120 3.5 3:a100c502:::rbd_data.1106b3d15a22.00000000000000cc:head [set-alloc-hint object_size 524288 write_size 524288,write 471040~9728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.603510+0000", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.11422468700000001, 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0042689249999999998, 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2120 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603510+0000", 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603510+0000", 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603508+0000", 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603532+0000", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4320000000000001e-05 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603555+0000", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2484999999999999e-05 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603566+0000", 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1491000000000001e-05 2024-09-19T21:34:37.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603652+0000", 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5357000000000003e-05 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.603853+0000", 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200974 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 3", 2024-09-19T21:34:37.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.604026+0000", 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017366500000000001 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.605175+0000", 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001148949 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.607652+0000", 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024769789999999998 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.607718+0000", 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5542000000000004e-05 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.607779+0000", 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1501000000000006e-05 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:2203 3.1d 3:b8c874ae:::rbd_data.1106b3d15a22.000000000000007c:head [set-alloc-hint object_size 524288 write_size 524288,write 176128~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.634284+0000", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.083450711999999996, 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050374840000000001, 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2203 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634284+0000", 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634284+0000", 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634282+0000", 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634303+0000", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1115e-05 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634326+0000", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2653000000000001e-05 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634336+0000", 2024-09-19T21:34:37.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0736e-05 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634403+0000", 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6713999999999999e-05 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634586+0000", 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018312100000000001 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.634750+0000", 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000163547 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.636541+0000", 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0017916099999999999 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639205+0000", 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026637359999999999 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639273+0000", 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8164000000000007e-05 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639322+0000", 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8566000000000002e-05 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:2210 3.e 3:71120a4d:::rbd_data.1106b3d15a22.000000000000016d:head [set-alloc-hint object_size 524288 write_size 524288,write 307712~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.639638+0000", 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.078097431999999994, 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029002049999999999, 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2210 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639638+0000", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639638+0000", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639636+0000", 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639643+0000", 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639655+0000", 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1887e-05 2024-09-19T21:34:37.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639664+0000", 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2819999999999996e-06 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639720+0000", 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6069999999999997e-05 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639838+0000", 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000118053 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.639941+0000", 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000103312 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.641922+0000", 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001980997 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.642456+0000", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00053408399999999999 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.642496+0000", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9606e-05 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.642538+0000", 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2033999999999998e-05 2024-09-19T21:34:37.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4512.0:2230 3.e 3:71120a4d:::rbd_data.1106b3d15a22.000000000000016d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.649206+0000", 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.068529186000000006, 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0049128970000000003, 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4512", 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:37.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2230 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649206+0000", 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649206+0000", 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649205+0000", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649211+0000", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.742e-06 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649224+0000", 2024-09-19T21:34:37.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2781e-05 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649234+0000", 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7780000000000002e-06 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649300+0000", 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6210000000000005e-05 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649420+0000", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000120109 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.649573+0000", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000153258 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650215+0000", 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00064135699999999995 2024-09-19T21:34:37.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.654003+0000", 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003788479 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.654067+0000", 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3283999999999999e-05 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.654119+0000", 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2033999999999997e-05 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4512.0:2222 3.0 e29/23 3:0793f9c4:::rbd_data.1106b3d15a22.00000000000000cd:head v 29'7, mlcod=29'5)", 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:37.650222+0000", 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.067513398000000002, 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036097949999999998, 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:37.833 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650222+0000", 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650222+0000", 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:37.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650221+0000", 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650228+0000", 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7680000000000008e-06 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650237+0000", 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9109999999999999e-06 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650243+0000", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5500000000000002e-06 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650294+0000", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1162999999999998e-05 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.650321+0000", 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6465000000000002e-05 2024-09-19T21:34:37.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.653777+0000", 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034569290000000001 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:37.653831+0000", 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3900000000000002e-05 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:37.838 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:37.838 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:37.839 INFO:teuthology.orchestra.run.smithi122.stdout:254 write 0xa1b3a00 thru 0xa1bf80f (0xbe10 bytes) 2024-09-19T21:34:37.839 INFO:teuthology.orchestra.run.smithi122.stdout:255 read 0xa10e000 thru 0xa115140 (0x7141 bytes) 2024-09-19T21:34:37.840 INFO:teuthology.orchestra.run.smithi122.stdout:257 trunc from 0xa1bfa00 to 0x8fb9a00 2024-09-19T21:34:37.840 INFO:teuthology.orchestra.run.smithi122.stdout:259 read 0x3755600 thru 0x37585c3 (0x2fc4 bytes) 2024-09-19T21:34:37.840 INFO:teuthology.orchestra.run.smithi122.stdout:262 trunc from 0x8fb9a00 to 0x67fa600 2024-09-19T21:34:37.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:37.841 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:37.832+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:37.853 INFO:teuthology.orchestra.run.smithi122.stdout:264 read 0x2307000 thru 0x230fd5c (0x8d5d bytes) 2024-09-19T21:34:37.855 INFO:teuthology.orchestra.run.smithi122.stdout:266 punch from 0x5e5de00 to 0x5e67000, (0x9200 bytes) 2024-09-19T21:34:37.855 INFO:teuthology.orchestra.run.smithi122.stdout:267 read 0x476600 thru 0x47b8e4 (0x52e5 bytes) 2024-09-19T21:34:37.856 INFO:teuthology.orchestra.run.smithi122.stdout:270 read 0x1c1d600 thru 0x1c2bf66 (0xe967 bytes) 2024-09-19T21:34:37.856 INFO:teuthology.orchestra.run.smithi122.stdout:276 read 0x5e9a400 thru 0x5ea39f7 (0x95f8 bytes) 2024-09-19T21:34:37.856 INFO:teuthology.orchestra.run.smithi122.stdout:277 write 0x361e000 thru 0x362dd6c (0xfd6d bytes) 2024-09-19T21:34:37.857 INFO:teuthology.orchestra.run.smithi122.stdout:278 write 0x4d2b000 thru 0x4d3863e (0xd63f bytes) 2024-09-19T21:34:37.908 INFO:teuthology.orchestra.run.smithi122.stdout:279 write 0xe44fc00 thru 0xe454f55 (0x5356 bytes) 2024-09-19T21:34:37.909 INFO:teuthology.orchestra.run.smithi122.stdout:280 read 0x8013200 thru 0x801e282 (0xb083 bytes) 2024-09-19T21:34:37.910 INFO:teuthology.orchestra.run.smithi122.stdout:282 write 0x3620800 thru 0x362f98f (0xf190 bytes) 2024-09-19T21:34:37.911 INFO:teuthology.orchestra.run.smithi122.stdout:289 punch from 0xcbe0e00 to 0xcbe9e00, (0x9000 bytes) 2024-09-19T21:34:37.912 INFO:teuthology.orchestra.run.smithi122.stdout:290 write 0xa948e00 thru 0xa951f68 (0x9169 bytes) 2024-09-19T21:34:37.928 INFO:teuthology.orchestra.run.smithi122.stdout:291 trunc from 0xe455000 to 0x6a92e00 2024-09-19T21:34:37.939 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:37.932+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:37.959 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:37.959 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:37.959 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:37.959 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:37.968 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xe37ca00 2024-09-19T21:34:37.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:37.984 INFO:teuthology.orchestra.run.smithi122.stdout:292 trunc from 0x6a92e00 to 0xe37ca00 2024-09-19T21:34:38.004 INFO:teuthology.orchestra.run.smithi122.stdout:296 read 0x6ed9a00 thru 0x6ee3076 (0x9677 bytes) 2024-09-19T21:34:38.014 INFO:teuthology.orchestra.run.smithi122.stdout:297 trunc from 0xe37ca00 to 0x9bd0800 2024-09-19T21:34:38.033 INFO:teuthology.orchestra.run.smithi122.stdout:303 write 0x4f27600 thru 0x4f29b4b (0x254c bytes) 2024-09-19T21:34:38.037 INFO:teuthology.orchestra.run.smithi122.stdout:304 trunc from 0x9bd0800 to 0xb038a00 2024-09-19T21:34:38.040 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:38.032+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:38.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:34:38.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:38.062 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.2 to 1 (10000) 2024-09-19T21:34:38.067 INFO:teuthology.orchestra.run.smithi122.stdout:305 read 0x20f6000 thru 0x2104310 (0xe311 bytes) 2024-09-19T21:34:38.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-19T21:34:38.091 INFO:teuthology.orchestra.run.smithi122.stdout:306 write 0xdb18800 thru 0xdb1dc92 (0x5493 bytes) 2024-09-19T21:34:38.091 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:38.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:38.092 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:38.092 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:38.092 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:38.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:38.127 INFO:teuthology.orchestra.run.smithi122.stdout:307 trunc from 0xdb1de00 to 0x4f4000 2024-09-19T21:34:38.142 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:38.136+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:38.195 INFO:teuthology.orchestra.run.smithi122.stdout:309 punch from 0x10ca00 to 0x114600, (0x7c00 bytes) 2024-09-19T21:34:38.205 INFO:teuthology.orchestra.run.smithi122.stdout:311 trunc from 0x4f4000 to 0x34f8400 2024-09-19T21:34:38.206 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:38.220 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:38.220 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:38.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:38.220 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:269 3.2 3:45d0fe23:::rbd_data.1106b3d15a22.0000000000000166:head [set-alloc-hint object_size 524288 write_size 524288,write 303104~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.517338971, 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037779380000000002, 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 269 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698826+0000", 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698865+0000", 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9332999999999998e-05 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698901+0000", 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5295e-05 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698916+0000", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5637e-05 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699098+0000", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181996 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699367+0000", 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026868699999999998 2024-09-19T21:34:38.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699677+0000", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031034299999999998 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.701550+0000", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018722280000000001 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702379+0000", 2024-09-19T21:34:38.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00082975300000000002 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702506+0000", 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683700000000001 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702606+0000", 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9951999999999994e-05 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:298 3.2 3:45d0fe23:::rbd_data.1106b3d15a22.0000000000000166:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.497530786, 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046917759999999999, 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-19T21:34:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718635+0000", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718645+0000", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9380000000000004e-06 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718669+0000", 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4635999999999999e-05 2024-09-19T21:34:38.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718683+0000", 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.33e-05 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718751+0000", 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8713000000000004e-05 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718856+0000", 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010454999999999999 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.719001+0000", 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014516999999999999 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.720036+0000", 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010348149999999999 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723209+0000", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031734049999999998 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723279+0000", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9465e-05 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723328+0000", 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9249999999999998e-05 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:408 3.1b 3:dce135ad:::rbd_data.1106b3d15a22.0000000000000058:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 370688~1536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.398181264, 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049773079999999997, 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 408 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817983+0000", 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817999+0000", 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5979000000000001e-05 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818029+0000", 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9793000000000001e-05 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818043+0000", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4229999999999999e-05 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818137+0000", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3790999999999997e-05 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818370+0000", 2024-09-19T21:34:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023274799999999999 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818704+0000", 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00033422299999999997 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.820107+0000", 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001402913 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822684+0000", 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025769899999999999 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822843+0000", 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015965 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822963+0000", 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011988400000000001 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:506 3.b 3:d3dc4512:::rbd_data.1106b3d15a22.000000000000007a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 311296~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.311511254, 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040462179999999999, 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 506 2024-09-19T21:34:38.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904653+0000", 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904730+0000", 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6845e-05 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904772+0000", 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1749000000000003e-05 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904792+0000", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0876000000000001e-05 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904964+0000", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000171473 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.905265+0000", 2024-09-19T21:34:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030149000000000002 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.905686+0000", 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042071399999999998 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.907748+0000", 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020616979999999998 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908546+0000", 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079803000000000003 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908634+0000", 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8623999999999997e-05 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908702+0000", 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7662000000000004e-05 2024-09-19T21:34:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:520 3.1b 3:dea6bd93:::rbd_data.1106b3d15a22.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.265365262, 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039707199999999996, 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 520 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950800+0000", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950812+0000", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1939e-05 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950859+0000", 2024-09-19T21:34:38.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6652e-05 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950894+0000", 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5076000000000001e-05 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951052+0000", 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015787 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951257+0000", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020509 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951403+0000", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001461 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.953520+0000", 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021174689999999999 2024-09-19T21:34:38.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954591+0000", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010710959999999999 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954668+0000", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6268999999999998e-05 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954773+0000", 2024-09-19T21:34:38.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105052 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:522 3.1b 3:dce135ad:::rbd_data.1106b3d15a22.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2651464450000001, 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064470990000000004, 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 522 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951019+0000", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951030+0000", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0908e-05 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951054+0000", 2024-09-19T21:34:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3862999999999998e-05 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951069+0000", 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5404999999999999e-05 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952281+0000", 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012111279999999999 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952524+0000", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024398099999999999 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952820+0000", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029595100000000003 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954401+0000", 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015803600000000001 2024-09-19T21:34:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957308+0000", 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029067950000000002 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957395+0000", 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7347000000000002e-05 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957468+0000", 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-05 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:527 3.b 3:d3dc4512:::rbd_data.1106b3d15a22.000000000000007a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.265066096, 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006342739, 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.257 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 527 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:38.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951100+0000", 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951109+0000", 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4399999999999994e-06 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951127+0000", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7694e-05 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951140+0000", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2883e-05 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952221+0000", 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001080805 2024-09-19T21:34:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952486+0000", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000265751 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952646+0000", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000159111 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.953836+0000", 2024-09-19T21:34:38.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001190633 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957259+0000", 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003422666 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957357+0000", 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8080999999999998e-05 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957444+0000", 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6821000000000001e-05 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:684 3.18 3:1c05a35e:::rbd_data.1106b3d15a22.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 278016~16896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.140781099, 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034127929999999999, 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 684 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075385+0000", 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075409+0000", 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4655999999999999e-05 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075432+0000", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2416999999999999e-05 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075442+0000", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9159999999999996e-06 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075582+0000", 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014003799999999999 2024-09-19T21:34:38.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.076012+0000", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000430424 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.076619+0000", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00060653300000000003 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078494+0000", 2024-09-19T21:34:38.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018752370000000001 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078717+0000", 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022346200000000001 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078765+0000", 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8034999999999997e-05 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078799+0000", 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3611000000000001e-05 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:782 3.1b 3:dec1c4c2:::rbd_data.1106b3d15a22.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 503808~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.0537167160000001, 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048048689999999998, 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 782 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162449+0000", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162505+0000", 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6836000000000003e-05 2024-09-19T21:34:38.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162539+0000", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3995999999999998e-05 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162554+0000", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.467e-05 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162673+0000", 2024-09-19T21:34:38.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011909 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162928+0000", 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025465500000000001 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.163245+0000", 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031686000000000003 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.164884+0000", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001639684 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.166998+0000", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021132299999999998 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.167118+0000", 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120884 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.167255+0000", 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000136861 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:860 3.1e 3:78ad9967:::rbd_data.1106b3d15a22.000000000000003e:head [set-alloc-hint object_size 524288 write_size 524288,write 10752~13824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.94580092699999996, 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041940850000000002, 2024-09-19T21:34:38.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 860 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270364+0000", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270424+0000", 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9682000000000002e-05 2024-09-19T21:34:38.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270513+0000", 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9167999999999997e-05 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270546+0000", 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969000000000002e-05 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270740+0000", 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019459600000000001 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.271164+0000", 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042339400000000001 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.271590+0000", 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042604999999999997 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274248+0000", 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026580390000000001 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274269+0000", 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.09e-05 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274354+0000", 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5284999999999994e-05 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274560+0000", 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020618200000000001 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2178 3.b 3:d5c01908:::rbd_data.1106b3d15a22.00000000000000bb:head [set-alloc-hint object_size 524288 write_size 524288,write 152576~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:38.281 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.58584276199999996, 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035989860000000002, 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2178 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:38.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630322+0000", 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630335+0000", 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3230999999999999e-05 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630363+0000", 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8031e-05 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630383+0000", 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9562e-05 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630505+0000", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121976 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630775+0000", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.631084+0000", 2024-09-19T21:34:38.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000308881 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.632548+0000", 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001464202 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633795+0000", 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012469429999999999 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633868+0000", 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3267000000000002e-05 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633923+0000", 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.507e-05 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2338 3.b 3:d5242dab:::rbd_data.1106b3d15a22.000000000000014c:head [set-alloc-hint object_size 524288 write_size 524288,write 180224~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.49579410600000001, 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040866779999999998, 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2338 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720371+0000", 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720401+0000", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0273e-05 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720436+0000", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4749000000000002e-05 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720451+0000", 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.49e-05 2024-09-19T21:34:38.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720572+0000", 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120907 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720814+0000", 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024252799999999999 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.721040+0000", 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022543199999999999 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.722832+0000", 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017927139999999999 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724286+0000", 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014541879999999999 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724375+0000", 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8887999999999993e-05 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724460+0000", 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4375000000000004e-05 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2572 3.1b 3:d8f57db8:::rbd_data.1106b3d15a22.00000000000000bc:head [set-alloc-hint object_size 524288 write_size 524288,write 192512~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.35653152799999999, 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035843649999999999, 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2572 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859633+0000", 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859691+0000", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7685000000000001e-05 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859726+0000", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5083000000000003e-05 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859746+0000", 2024-09-19T21:34:38.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713e-05 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859905+0000", 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015812899999999999 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.860240+0000", 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00033541499999999999 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.860338+0000", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7915e-05 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.861685+0000", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013472149999999999 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863165+0000", 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014794859999999999 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863197+0000", 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2551000000000001e-05 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863220+0000", 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2895999999999999e-05 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2785 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.210659132, 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033518430000000002, 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2785 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005510+0000", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2199999999999999e-06 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005515+0000", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2379999999999997e-06 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005521+0000", 2024-09-19T21:34:38.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008672+0000", 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031510560000000002 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008751+0000", 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8557e-05 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008860+0000", 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010899 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2975 3.1b 3:dec1c4c2:::rbd_data.1106b3d15a22.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.068334918999999994, 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053820750000000001, 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2975 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147830+0000", 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147843+0000", 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3373000000000001e-05 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147871+0000", 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7759e-05 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147887+0000", 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6356000000000001e-05 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148013+0000", 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000125866 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148244+0000", 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023111 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148490+0000", 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024625599999999998 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.149553+0000", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001062269 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.152898+0000", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033457529999999999 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153128+0000", 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022958899999999999 2024-09-19T21:34:38.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153214+0000", 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6286999999999996e-05 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4512.0:3045 3.7 e30/23 3:e05ea290:::rbd_data.1106b3d15a22.000000000000008c:head v 30'8, mlcod=28'6)", 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.058485283999999998, 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034570489999999998, 2024-09-19T21:34:38.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157681+0000", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157689+0000", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4909999999999999e-06 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157704+0000", 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5438999999999999e-05 2024-09-19T21:34:38.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157711+0000", 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8639999999999998e-06 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157779+0000", 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8326999999999998e-05 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157818+0000", 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9236999999999997e-05 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161091+0000", 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003272423 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161139+0000", 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8075000000000003e-05 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4512.0:3052 3.15 e30/23 3:a953a98d:::rbd_data.1106b3d15a22.000000000000009b:head v 30'8, mlcod=29'6)", 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.057619366999999998, 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918860000000003, 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158547+0000", 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158558+0000", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899e-05 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158621+0000", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3660999999999999e-05 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158641+0000", 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9873000000000001e-05 2024-09-19T21:34:38.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158730+0000", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9036999999999995e-05 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158750+0000", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9743000000000001e-05 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163593+0000", 2024-09-19T21:34:38.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048436219999999997 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163640+0000", 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6189000000000001e-05 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3057 3.1e 3:7eeb20d1:::rbd_data.1106b3d15a22.000000000000009f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.056946638000000001, 2024-09-19T21:34:38.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045117480000000003, 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3057 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159219+0000", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159225+0000", 2024-09-19T21:34:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159239+0000", 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3372e-05 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159245+0000", 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.054e-06 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159310+0000", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5717999999999997e-05 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159411+0000", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100975 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159530+0000", 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118274 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.160656+0000", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011268459999999999 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163623+0000", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002966794 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163686+0000", 2024-09-19T21:34:38.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3107000000000004e-05 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163732+0000", 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5976999999999997e-05 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3060 3.b 3:d5c01908:::rbd_data.1106b3d15a22.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:38.325 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.056659866000000003, 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004251052, 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3060 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159510+0000", 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7019999999999999e-06 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159517+0000", 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1380000000000004e-06 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159521+0000", 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0559999999999998e-06 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159573+0000", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-05 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159690+0000", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116774 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159832+0000", 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014233299999999999 2024-09-19T21:34:38.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.160830+0000", 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0009977059999999999 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163663+0000", 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002832401 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163704+0000", 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1724999999999998e-05 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163758+0000", 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029000000000001e-05 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3063 3.1b 3:d8f57db8:::rbd_data.1106b3d15a22.00000000000000bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:38.331 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.054727252999999997, 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049226620000000004, 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3063 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161439+0000", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161445+0000", 2024-09-19T21:34:38.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9020000000000004e-06 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161458+0000", 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2462e-05 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161463+0000", 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9559999999999996e-06 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161516+0000", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3955000000000001e-05 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161609+0000", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2647999999999998e-05 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161735+0000", 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012548799999999999 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163191+0000", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001455954 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166285+0000", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003094336 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166330+0000", 2024-09-19T21:34:38.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4972999999999998e-05 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166363+0000", 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2670999999999999e-05 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:38.337 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:38.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:38.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:38.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:38.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:38.340 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:38.332+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:38.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:38.341 INFO:teuthology.orchestra.run.smithi122.stdout:313 read 0x29b8400 thru 0x29bf536 (0x7137 bytes) 2024-09-19T21:34:38.341 INFO:teuthology.orchestra.run.smithi122.stdout:315 read 0x2a8da00 thru 0x2a9cf6c (0xf56d bytes) 2024-09-19T21:34:38.341 INFO:teuthology.orchestra.run.smithi122.stdout:316 punch from 0xb01e00 to 0xb02800, (0xa00 bytes) 2024-09-19T21:34:38.341 INFO:teuthology.orchestra.run.smithi122.stdout:319 trunc from 0x34f8400 to 0x6ac9e00 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:322 read 0x2d68e00 thru 0x2d737cf (0xa9d0 bytes) 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:323 read 0x239f000 thru 0x23a6781 (0x7782 bytes) 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:324 trunc from 0x6ac9e00 to 0x64da000 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:325 read 0x1611800 thru 0x16191b3 (0x79b4 bytes) 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:326 read 0x1ba0a00 thru 0x1ba8756 (0x7d57 bytes) 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:327 write 0x3a64a00 thru 0x3a67b0f (0x3110 bytes) 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:328 trunc from 0x64da000 to 0xd93ce00 2024-09-19T21:34:38.342 INFO:teuthology.orchestra.run.smithi122.stdout:329 trunc from 0xd93ce00 to 0x2b1f800 2024-09-19T21:34:38.343 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:38.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:38.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:38.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:38.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:38.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:38.347 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:38.364 INFO:teuthology.orchestra.run.smithi122.stdout:330 read 0x23f4000 thru 0x23fe333 (0xa334 bytes) 2024-09-19T21:34:38.366 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xe9af200 2024-09-19T21:34:38.388 INFO:teuthology.orchestra.run.smithi122.stdout:332 trunc from 0x2b1f800 to 0xe9af200 2024-09-19T21:34:38.405 INFO:teuthology.orchestra.run.smithi122.stdout:334 write 0x22bd800 thru 0x22c77e0 (0x9fe1 bytes) 2024-09-19T21:34:38.406 INFO:teuthology.orchestra.run.smithi122.stdout:335 punch from 0xb426400 to 0xb42fe00, (0x9a00 bytes) 2024-09-19T21:34:38.407 INFO:teuthology.orchestra.run.smithi122.stdout:336 write 0x42e000 thru 0x438dcc (0xadcd bytes) 2024-09-19T21:34:38.407 INFO:teuthology.orchestra.run.smithi122.stdout:337 read 0xfdd200 thru 0xfe299a (0x579b bytes) 2024-09-19T21:34:38.407 INFO:teuthology.orchestra.run.smithi122.stdout:338 write 0x884f200 thru 0x8858b22 (0x9923 bytes) 2024-09-19T21:34:38.407 INFO:teuthology.orchestra.run.smithi122.stdout:339 punch from 0x96b0000 to 0x96b5000, (0x5000 bytes) 2024-09-19T21:34:38.409 INFO:teuthology.orchestra.run.smithi122.stdout:342 write 0x7a6bc00 thru 0x7a73808 (0x7c09 bytes) 2024-09-19T21:34:38.432 INFO:teuthology.orchestra.run.smithi122.stdout:343 trunc from 0xe9af200 to 0x26dfc00 2024-09-19T21:34:38.439 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:38.434+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:38.459 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:38.459 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:38.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:38.489 INFO:teuthology.orchestra.run.smithi122.stdout:344 write 0x6352600 thru 0x63615e9 (0xefea bytes) 2024-09-19T21:34:38.490 INFO:teuthology.orchestra.run.smithi122.stdout:347 read 0x4c53e00 thru 0x4c602a2 (0xc4a3 bytes) 2024-09-19T21:34:38.504 INFO:teuthology.orchestra.run.smithi122.stdout:348 trunc from 0x6361600 to 0xcd72a00 2024-09-19T21:34:38.528 INFO:teuthology.orchestra.run.smithi122.stdout:349 write 0xc2d7e00 thru 0xc2da26e (0x246f bytes) 2024-09-19T21:34:38.531 INFO:teuthology.orchestra.run.smithi122.stdout:352 trunc from 0xcd72a00 to 0xc67b600 2024-09-19T21:34:38.540 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:38.532+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:38.553 INFO:teuthology.orchestra.run.smithi122.stdout:355 write 0x24e0200 thru 0x24e18dd (0x16de bytes) 2024-09-19T21:34:38.554 INFO:teuthology.orchestra.run.smithi122.stdout:356 read 0x2fc5400 thru 0x2fc72f9 (0x1efa bytes) 2024-09-19T21:34:38.554 INFO:teuthology.orchestra.run.smithi122.stdout:362 punch from 0xa099e00 to 0xa0a8000, (0xe200 bytes) 2024-09-19T21:34:38.554 INFO:teuthology.orchestra.run.smithi122.stdout:363 punch from 0x7b30a00 to 0x7b38a00, (0x8000 bytes) 2024-09-19T21:34:38.555 INFO:teuthology.orchestra.run.smithi122.stdout:364 write 0x546ba00 thru 0x5477928 (0xbf29 bytes) 2024-09-19T21:34:38.555 INFO:teuthology.orchestra.run.smithi122.stdout:366 read 0x5299c00 thru 0x52a372a (0x9b2b bytes) 2024-09-19T21:34:38.555 INFO:teuthology.orchestra.run.smithi122.stdout:367 read 0xaf24400 thru 0xaf2e2bf (0x9ec0 bytes) 2024-09-19T21:34:38.556 INFO:teuthology.orchestra.run.smithi122.stdout:369 punch from 0x67f4200 to 0x67f5600, (0x1400 bytes) 2024-09-19T21:34:38.557 INFO:teuthology.orchestra.run.smithi122.stdout:370 read 0x413200 thru 0x415aff (0x2900 bytes) 2024-09-19T21:34:38.561 INFO:teuthology.orchestra.run.smithi122.stdout:372 read 0x1b7ee00 thru 0x1b883fe (0x95ff bytes) 2024-09-19T21:34:38.561 INFO:teuthology.orchestra.run.smithi122.stdout:373 read 0x3331000 thru 0x334090b (0xf90c bytes) 2024-09-19T21:34:38.562 INFO:teuthology.orchestra.run.smithi122.stdout:374 write 0xc25a400 thru 0xc25fc49 (0x584a bytes) 2024-09-19T21:34:38.563 INFO:teuthology.orchestra.run.smithi122.stdout:375 read 0x8fce000 thru 0x8fd0f0c (0x2f0d bytes) 2024-09-19T21:34:38.582 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:38.582 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:38.589 INFO:teuthology.orchestra.run.smithi122.stdout:379 write 0xe86be00 thru 0xe87395a (0x7b5b bytes) 2024-09-19T21:34:38.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:38.592 INFO:teuthology.orchestra.run.smithi122.stdout:380 read 0x9ccba00 thru 0x9cd3c47 (0x8248 bytes) 2024-09-19T21:34:38.604 INFO:teuthology.orchestra.run.smithi122.stdout:382 trunc from 0xe873a00 to 0x8c01a00 2024-09-19T21:34:38.641 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:38.638+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:38.653 INFO:teuthology.orchestra.run.smithi122.stdout:383 write 0x5919400 thru 0x5925eb2 (0xcab3 bytes) 2024-09-19T21:34:38.658 INFO:teuthology.orchestra.run.smithi122.stdout:384 trunc from 0x8c01a00 to 0x97e8400 2024-09-19T21:34:38.698 INFO:teuthology.orchestra.run.smithi122.stdout:385 trunc from 0x97e8400 to 0x42b5400 2024-09-19T21:34:38.699 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:38.699 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:38.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:38.736 INFO:teuthology.orchestra.run.smithi122.stdout:387 trunc from 0x42b5400 to 0x6012400 2024-09-19T21:34:38.741 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:38.738+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:38.755 INFO:teuthology.orchestra.run.smithi122.stdout:388 write 0x1cc4800 thru 0x1cceb2f (0xa330 bytes) 2024-09-19T21:34:38.768 INFO:teuthology.orchestra.run.smithi122.stdout:389 trunc from 0x6012400 to 0xa167800 2024-09-19T21:34:38.820 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:38.821 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:38.824 INFO:teuthology.orchestra.run.smithi122.stdout:390 write 0xabfc800 thru 0xac00bd4 (0x43d5 bytes) 2024-09-19T21:34:38.825 INFO:teuthology.orchestra.run.smithi122.stdout:391 read 0x35f3400 thru 0x35f49f6 (0x15f7 bytes) 2024-09-19T21:34:38.826 INFO:teuthology.orchestra.run.smithi122.stdout:392 read 0xd7f200 thru 0xd86059 (0x6e5a bytes) 2024-09-19T21:34:38.826 INFO:teuthology.orchestra.run.smithi122.stdout:393 read 0x4851800 thru 0x485e38c (0xcb8d bytes) 2024-09-19T21:34:38.826 INFO:teuthology.orchestra.run.smithi122.stdout:396 write 0x5e59e00 thru 0x5e5df97 (0x4198 bytes) 2024-09-19T21:34:38.827 INFO:teuthology.orchestra.run.smithi122.stdout:399 punch from 0x9b42200 to 0x9b45000, (0x2e00 bytes) 2024-09-19T21:34:38.827 INFO:teuthology.orchestra.run.smithi122.stdout:400 read 0x32d200 thru 0x33025b (0x305c bytes) 2024-09-19T21:34:38.828 INFO:teuthology.orchestra.run.smithi122.stdout:402 write 0x548b000 thru 0x548efdf (0x3fe0 bytes) 2024-09-19T21:34:38.828 INFO:teuthology.orchestra.run.smithi122.stdout:404 read 0x4753400 thru 0x4758810 (0x5411 bytes) 2024-09-19T21:34:38.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:38.864 INFO:teuthology.orchestra.run.smithi122.stdout:405 write 0xbcd6a00 thru 0xbce5ff7 (0xf5f8 bytes) 2024-09-19T21:34:38.866 INFO:teuthology.orchestra.run.smithi122.stdout:406 write 0x7052a00 thru 0x705db37 (0xb138 bytes) 2024-09-19T21:34:38.866 INFO:teuthology.orchestra.run.smithi122.stdout:407 punch from 0x2e05a00 to 0x2e10a00, (0xb000 bytes) 2024-09-19T21:34:38.868 INFO:teuthology.orchestra.run.smithi122.stdout:408 read 0x792d400 thru 0x793ae8a (0xda8b bytes) 2024-09-19T21:34:38.880 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:38.878+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:38.893 INFO:teuthology.orchestra.run.smithi122.stdout:410 write 0xc965e00 thru 0xc970032 (0xa233 bytes) 2024-09-19T21:34:38.898 INFO:teuthology.orchestra.run.smithi122.stdout:411 trunc from 0xc970200 to 0xb2a7200 2024-09-19T21:34:38.932 INFO:teuthology.orchestra.run.smithi122.stdout:413 read 0x9ded200 thru 0x9df3e9e (0x6c9f bytes) 2024-09-19T21:34:38.944 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:38.938+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:38.948 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:38.948 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:38.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:38.959 INFO:teuthology.orchestra.run.smithi122.stdout:415 write 0xc384c00 thru 0xc388ae7 (0x3ee8 bytes) 2024-09-19T21:34:38.960 INFO:teuthology.orchestra.run.smithi122.stdout:416 read 0x2766200 thru 0x276aea5 (0x4ca6 bytes) 2024-09-19T21:34:38.961 INFO:teuthology.orchestra.run.smithi122.stdout:418 read 0x6a4b200 thru 0x6a4f862 (0x4663 bytes) 2024-09-19T21:34:38.961 INFO:teuthology.orchestra.run.smithi122.stdout:420 write 0x53c1000 thru 0x53cf5ae (0xe5af bytes) 2024-09-19T21:34:38.999 INFO:teuthology.orchestra.run.smithi122.stdout:421 write 0xe92a600 thru 0xe92da13 (0x3414 bytes) 2024-09-19T21:34:39.000 INFO:teuthology.orchestra.run.smithi122.stdout:423 read 0xdf76000 thru 0xdf82637 (0xc638 bytes) 2024-09-19T21:34:39.000 INFO:teuthology.orchestra.run.smithi122.stdout:425 punch from 0x3448400 to 0x3453400, (0xb000 bytes) 2024-09-19T21:34:39.001 INFO:teuthology.orchestra.run.smithi122.stdout:430 clone 2 order 19 su 2048 sc 9 2024-09-19T21:34:39.045 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:39.042+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:39.069 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.069 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.074 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:34:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-09-19T21:34:39.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:39.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-19T21:34:39.145 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:39.142+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:39.188 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.188 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:39.246 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:39.242+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:39.301 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.301 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:39.347 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:39.340+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:39.420 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.421 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.430 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:39.448 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:39.442+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:39.537 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.537 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:39.549 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:39.540+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:39.649 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:39.640+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:39.658 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.658 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.669 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:39.750 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:39.746+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:39.849 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-19T21:34:39.850 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:39.844+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:39.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:39.994 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:39.988+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:40.051 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:40.046+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:40.076 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-09-19T21:34:40.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-19T21:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:40.124 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:40.170 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:40.164+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:40.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:40.253 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:40.244+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:40.354 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:40.348+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:40.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:40.365 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:40.365 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:40.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:40.455 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:40.448+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:40.487 INFO:teuthology.orchestra.run.smithi122.stdout:431 write 0x6a31200 thru 0x6a3bd21 (0xab22 bytes) 2024-09-19T21:34:40.488 INFO:teuthology.orchestra.run.smithi122.stdout:432 write 0x69b0000 thru 0x69b9904 (0x9905 bytes) 2024-09-19T21:34:40.494 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:40.494 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:40.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:40.556 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:40.548+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:40.562 INFO:teuthology.orchestra.run.smithi122.stdout:433 trunc from 0xe92dc00 to 0x114e600 2024-09-19T21:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:40.614 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:40.615 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:269 3.2 3:45d0fe23:::rbd_data.1106b3d15a22.0000000000000166:head [set-alloc-hint object_size 524288 write_size 524288,write 303104~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.930042179, 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0037779380000000002, 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 269 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698828+0000", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698826+0000", 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:40.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698865+0000", 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9332999999999998e-05 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698901+0000", 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5295e-05 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.698916+0000", 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5637e-05 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699098+0000", 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000181996 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699367+0000", 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00026868699999999998 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.699677+0000", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031034299999999998 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.701550+0000", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018722280000000001 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702379+0000", 2024-09-19T21:34:40.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00082975300000000002 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702506+0000", 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683700000000001 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.702606+0000", 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9951999999999994e-05 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:298 3.2 3:45d0fe23:::rbd_data.1106b3d15a22.0000000000000166:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9102339939999999, 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0046917759999999999, 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 298 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718636+0000", 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718635+0000", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718645+0000", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9380000000000004e-06 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718669+0000", 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4635999999999999e-05 2024-09-19T21:34:40.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718683+0000", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.33e-05 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718751+0000", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8713000000000004e-05 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.718856+0000", 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010454999999999999 2024-09-19T21:34:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.719001+0000", 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014516999999999999 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.720036+0000", 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010348149999999999 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723209+0000", 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031734049999999998 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723279+0000", 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9465e-05 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.723328+0000", 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9249999999999998e-05 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:408 3.1b 3:dce135ad:::rbd_data.1106b3d15a22.0000000000000058:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 370688~1536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.8108844720000001, 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049773079999999997, 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 408 2024-09-19T21:34:40.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817986+0000", 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817983+0000", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.817999+0000", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5979000000000001e-05 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818029+0000", 2024-09-19T21:34:40.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9793000000000001e-05 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818043+0000", 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4229999999999999e-05 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818137+0000", 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3790999999999997e-05 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818370+0000", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023274799999999999 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.818704+0000", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00033422299999999997 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.820107+0000", 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001402913 2024-09-19T21:34:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822684+0000", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0025769899999999999 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822843+0000", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015965 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.822963+0000", 2024-09-19T21:34:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011988400000000001 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:506 3.b 3:d3dc4512:::rbd_data.1106b3d15a22.000000000000007a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 311296~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7242144619999999, 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040462179999999999, 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.651 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 506 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904656+0000", 2024-09-19T21:34:40.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904653+0000", 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904730+0000", 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6845e-05 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904772+0000", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1749000000000003e-05 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904792+0000", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0876000000000001e-05 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.904964+0000", 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000171473 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.905265+0000", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030149000000000002 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.905686+0000", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042071399999999998 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.907748+0000", 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0020616979999999998 2024-09-19T21:34:40.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908546+0000", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00079803000000000003 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908634+0000", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8623999999999997e-05 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.908702+0000", 2024-09-19T21:34:40.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7662000000000004e-05 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:520 3.1b 3:dea6bd93:::rbd_data.1106b3d15a22.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.6780684699999999, 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039707199999999996, 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 520 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950802+0000", 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950800+0000", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950812+0000", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1939e-05 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950859+0000", 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6652e-05 2024-09-19T21:34:40.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.950894+0000", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5076000000000001e-05 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951052+0000", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015787 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951257+0000", 2024-09-19T21:34:40.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020509 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951403+0000", 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0001461 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.953520+0000", 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021174689999999999 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954591+0000", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010710959999999999 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954668+0000", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6268999999999998e-05 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954773+0000", 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105052 2024-09-19T21:34:40.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:522 3.1b 3:dce135ad:::rbd_data.1106b3d15a22.0000000000000058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.677849653, 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064470990000000004, 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 522 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951021+0000", 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951019+0000", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951030+0000", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0908e-05 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951054+0000", 2024-09-19T21:34:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3862999999999998e-05 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951069+0000", 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5404999999999999e-05 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952281+0000", 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012111279999999999 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952524+0000", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024398099999999999 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952820+0000", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029595100000000003 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.954401+0000", 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015803600000000001 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957308+0000", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0029067950000000002 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957395+0000", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7347000000000002e-05 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957468+0000", 2024-09-19T21:34:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2869999999999999e-05 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:527 3.b 3:d3dc4512:::rbd_data.1106b3d15a22.000000000000007a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.6777693039999999, 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006342739, 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 527 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951101+0000", 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951100+0000", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951109+0000", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4399999999999994e-06 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951127+0000", 2024-09-19T21:34:40.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7694e-05 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.951140+0000", 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2883e-05 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952221+0000", 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001080805 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952486+0000", 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000265751 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.952646+0000", 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000159111 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.953836+0000", 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001190633 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957259+0000", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003422666 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957357+0000", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8080999999999998e-05 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:36.957444+0000", 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6821000000000001e-05 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:684 3.18 3:1c05a35e:::rbd_data.1106b3d15a22.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 278016~16896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.5534843069999997, 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034127929999999999, 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 684 2024-09-19T21:34:40.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075386+0000", 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075385+0000", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075409+0000", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4655999999999999e-05 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075432+0000", 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2416999999999999e-05 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075442+0000", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9159999999999996e-06 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.075582+0000", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014003799999999999 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.076012+0000", 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000430424 2024-09-19T21:34:40.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 4", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.076619+0000", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00060653300000000003 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078494+0000", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0018752370000000001 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078717+0000", 2024-09-19T21:34:40.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022346200000000001 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078765+0000", 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8034999999999997e-05 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.078799+0000", 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3611000000000001e-05 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:782 3.1b 3:dec1c4c2:::rbd_data.1106b3d15a22.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 503808~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.4664199240000002, 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048048689999999998, 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 782 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162450+0000", 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162449+0000", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162505+0000", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6836000000000003e-05 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162539+0000", 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3995999999999998e-05 2024-09-19T21:34:40.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162554+0000", 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.467e-05 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162673+0000", 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011909 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.162928+0000", 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025465500000000001 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.163245+0000", 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031686000000000003 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.164884+0000", 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001639684 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.166998+0000", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021132299999999998 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.167118+0000", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120884 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.167255+0000", 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000136861 2024-09-19T21:34:40.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:860 3.1e 3:78ad9967:::rbd_data.1106b3d15a22.000000000000003e:head [set-alloc-hint object_size 524288 write_size 524288,write 10752~13824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.358504135, 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0041940850000000002, 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 860 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270366+0000", 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270364+0000", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270424+0000", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9682000000000002e-05 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270513+0000", 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9167999999999997e-05 2024-09-19T21:34:40.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270546+0000", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2969000000000002e-05 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.270740+0000", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019459600000000001 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.271164+0000", 2024-09-19T21:34:40.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042339400000000001 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.271590+0000", 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00042604999999999997 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274248+0000", 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0026580390000000001 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274269+0000", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.09e-05 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274354+0000", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5284999999999994e-05 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.274560+0000", 2024-09-19T21:34:40.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020618200000000001 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2178 3.b 3:d5c01908:::rbd_data.1106b3d15a22.00000000000000bb:head [set-alloc-hint object_size 524288 write_size 524288,write 152576~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.9985459699999999, 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035989860000000002, 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2178 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630324+0000", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630322+0000", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630335+0000", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3230999999999999e-05 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630363+0000", 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8031e-05 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630383+0000", 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9562e-05 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630505+0000", 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121976 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.630775+0000", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.631084+0000", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000308881 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.632548+0000", 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001464202 2024-09-19T21:34:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633795+0000", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0012469429999999999 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633868+0000", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3267000000000002e-05 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.633923+0000", 2024-09-19T21:34:40.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.507e-05 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2338 3.b 3:d5242dab:::rbd_data.1106b3d15a22.000000000000014c:head [set-alloc-hint object_size 524288 write_size 524288,write 180224~14336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.9084973139999999, 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0040866779999999998, 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2338 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720373+0000", 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720371+0000", 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720401+0000", 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0273e-05 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720436+0000", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4749000000000002e-05 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720451+0000", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.49e-05 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720572+0000", 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120907 2024-09-19T21:34:40.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.720814+0000", 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024252799999999999 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.721040+0000", 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022543199999999999 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.722832+0000", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0017927139999999999 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724286+0000", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014541879999999999 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724375+0000", 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8887999999999993e-05 2024-09-19T21:34:40.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.724460+0000", 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4375000000000004e-05 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2572 3.1b 3:d8f57db8:::rbd_data.1106b3d15a22.00000000000000bc:head [set-alloc-hint object_size 524288 write_size 524288,write 192512~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.769234736, 2024-09-19T21:34:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035843649999999999, 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2572 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859636+0000", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859633+0000", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859691+0000", 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7685000000000001e-05 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859726+0000", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5083000000000003e-05 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859746+0000", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0713e-05 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.859905+0000", 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015812899999999999 2024-09-19T21:34:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.860240+0000", 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00033541499999999999 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.860338+0000", 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7915e-05 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.861685+0000", 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0013472149999999999 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863165+0000", 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0014794859999999999 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863197+0000", 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2551000000000001e-05 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.863220+0000", 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2895999999999999e-05 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2785 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.6233623399999999, 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033518430000000002, 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2785 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005508+0000", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005510+0000", 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2199999999999999e-06 2024-09-19T21:34:40.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005515+0000", 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2379999999999997e-06 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.005521+0000", 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008672+0000", 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031510560000000002 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008751+0000", 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8557e-05 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.008860+0000", 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010899 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.715 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2975 3.1b 3:dec1c4c2:::rbd_data.1106b3d15a22.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.4810381270000001, 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053820750000000001, 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2975 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.716 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147832+0000", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147830+0000", 2024-09-19T21:34:40.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147843+0000", 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3373000000000001e-05 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147871+0000", 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7759e-05 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.147887+0000", 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6356000000000001e-05 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148013+0000", 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000125866 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148244+0000", 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023111 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148490+0000", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024625599999999998 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.149553+0000", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001062269 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.152898+0000", 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0033457529999999999 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153128+0000", 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022958899999999999 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153214+0000", 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6286999999999996e-05 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4512.0:3045 3.7 e30/23 3:e05ea290:::rbd_data.1106b3d15a22.000000000000008c:head v 30'8, mlcod=28'6)", 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.471188492, 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0034570489999999998, 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157682+0000", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157681+0000", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157689+0000", 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4909999999999999e-06 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157704+0000", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5438999999999999e-05 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157711+0000", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8639999999999998e-06 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157779+0000", 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8326999999999998e-05 2024-09-19T21:34:40.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.157818+0000", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9236999999999997e-05 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161091+0000", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003272423 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161139+0000", 2024-09-19T21:34:40.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8075000000000003e-05 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4512.0:3052 3.15 e30/23 3:a953a98d:::rbd_data.1106b3d15a22.000000000000009b:head v 30'8, mlcod=29'6)", 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.470322575, 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0050918860000000003, 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158548+0000", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158547+0000", 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158558+0000", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0899e-05 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158621+0000", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3660999999999999e-05 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158641+0000", 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9873000000000001e-05 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158730+0000", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9036999999999995e-05 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.158750+0000", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9743000000000001e-05 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163593+0000", 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048436219999999997 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163640+0000", 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6189000000000001e-05 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3057 3.1e 3:7eeb20d1:::rbd_data.1106b3d15a22.000000000000009f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.4696498459999998, 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0045117480000000003, 2024-09-19T21:34:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3057 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159221+0000", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159219+0000", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159225+0000", 2024-09-19T21:34:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159239+0000", 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3372e-05 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159245+0000", 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.054e-06 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159310+0000", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5717999999999997e-05 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159411+0000", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100975 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159530+0000", 2024-09-19T21:34:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118274 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.160656+0000", 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011268459999999999 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163623+0000", 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002966794 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163686+0000", 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3107000000000004e-05 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163732+0000", 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5976999999999997e-05 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3060 3.b 3:d5c01908:::rbd_data.1106b3d15a22.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.4693630739999999, 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004251052, 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3060 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159507+0000", 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159510+0000", 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7019999999999999e-06 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159517+0000", 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1380000000000004e-06 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159521+0000", 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0559999999999998e-06 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159573+0000", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-05 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159690+0000", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116774 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.159832+0000", 2024-09-19T21:34:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014233299999999999 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.160830+0000", 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0009977059999999999 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163663+0000", 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.002832401 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163704+0000", 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1724999999999998e-05 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163758+0000", 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4029000000000001e-05 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:3063 3.1b 3:d8f57db8:::rbd_data.1106b3d15a22.00000000000000bc:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.4674304610000002, 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0049226620000000004, 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3063 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161440+0000", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161439+0000", 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161445+0000", 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9020000000000004e-06 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161458+0000", 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2462e-05 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161463+0000", 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9559999999999996e-06 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161516+0000", 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3955000000000001e-05 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161609+0000", 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2647999999999998e-05 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.161735+0000", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012548799999999999 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.163191+0000", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001455954 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166285+0000", 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.003094336 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166330+0000", 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4972999999999998e-05 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.166363+0000", 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2670999999999999e-05 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.748 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:40.751 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:40.751 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:40.746+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:436 trunc from 0x114e600 to 0xa526a00 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:437 trunc from 0xa526a00 to 0xc382000 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:438 read 0x487de00 thru 0x488ccf6 (0xeef7 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:439 read 0x49ddc00 thru 0x49e26cd (0x4ace bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:443 punch from 0xb1ca400 to 0xb1d8c00, (0xe800 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:446 write 0x8df3400 thru 0x8dffdb9 (0xc9ba bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:447 punch from 0x9652a00 to 0x9659e00, (0x7400 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:451 read 0x8dc2c00 thru 0x8dc68c3 (0x3cc4 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:452 write 0x6539c00 thru 0x65496e1 (0xfae2 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:457 punch from 0xaea3200 to 0xaeb1a00, (0xe800 bytes) 2024-09-19T21:34:40.752 INFO:teuthology.orchestra.run.smithi122.stdout:458 write 0x6fc000 thru 0x703d56 (0x7d57 bytes) 2024-09-19T21:34:40.753 INFO:teuthology.orchestra.run.smithi122.stdout:461 read 0x7ad7a00 thru 0x7ad9615 (0x1c16 bytes) 2024-09-19T21:34:40.753 INFO:teuthology.orchestra.run.smithi122.stdout:462 trunc from 0xc382000 to 0xa93e400 2024-09-19T21:34:40.817 INFO:teuthology.orchestra.run.smithi122.stdout:464 read 0x82800 thru 0x8a191 (0x7992 bytes) 2024-09-19T21:34:40.823 INFO:teuthology.orchestra.run.smithi122.stdout:466 write 0x4dad800 thru 0x4db8263 (0xaa64 bytes) 2024-09-19T21:34:40.851 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:40.846+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:40.892 INFO:teuthology.orchestra.run.smithi122.stdout:468 write 0xdf62c00 thru 0xdf69e02 (0x7203 bytes) 2024-09-19T21:34:40.894 INFO:teuthology.orchestra.run.smithi122.stdout:469 read 0xce33a00 thru 0xce33bb7 (0x1b8 bytes) 2024-09-19T21:34:40.894 INFO:teuthology.orchestra.run.smithi122.stdout:471 read 0xcf8b200 thru 0xcf8dc5d (0x2a5e bytes) 2024-09-19T21:34:40.895 INFO:teuthology.orchestra.run.smithi122.stdout:474 write 0xccbd800 thru 0xccc3ead (0x66ae bytes) 2024-09-19T21:34:40.895 INFO:teuthology.orchestra.run.smithi122.stdout:475 write 0x6b2b600 thru 0x6b38eaf (0xd8b0 bytes) 2024-09-19T21:34:40.895 INFO:teuthology.orchestra.run.smithi122.stdout:479 punch from 0xb9eda00 to 0xb9f2200, (0x4800 bytes) 2024-09-19T21:34:40.898 INFO:teuthology.orchestra.run.smithi122.stdout:481 read 0x2bf9800 thru 0x2bfc007 (0x2808 bytes) 2024-09-19T21:34:40.898 INFO:teuthology.orchestra.run.smithi122.stdout:484 write 0x7ca9e00 thru 0x7cb1477 (0x7678 bytes) 2024-09-19T21:34:40.898 INFO:teuthology.orchestra.run.smithi122.stdout:490 write 0x9b20000 thru 0x9b2bd5a (0xbd5b bytes) 2024-09-19T21:34:40.898 INFO:teuthology.orchestra.run.smithi122.stdout:491 write 0xf2b800 thru 0xf2cd50 (0x1551 bytes) 2024-09-19T21:34:40.899 INFO:teuthology.orchestra.run.smithi122.stdout:492 write 0x94f9800 thru 0x94fe76a (0x4f6b bytes) 2024-09-19T21:34:40.900 INFO:teuthology.orchestra.run.smithi122.stdout:496 read 0x1321200 thru 0x132ab0f (0x9910 bytes) 2024-09-19T21:34:40.900 INFO:teuthology.orchestra.run.smithi122.stdout:497 read 0x1deec00 thru 0x1dfc9ee (0xddef bytes) 2024-09-19T21:34:40.901 INFO:teuthology.orchestra.run.smithi122.stdout:498 read 0x705e00 thru 0x7064ae (0x6af bytes) 2024-09-19T21:34:40.902 INFO:teuthology.orchestra.run.smithi122.stdout:500 write 0x661d000 thru 0x66299f3 (0xc9f4 bytes) 2024-09-19T21:34:40.903 INFO:teuthology.orchestra.run.smithi122.stdout:501 clone 3 order 25 su 524288 sc 12 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2445 3.7 3:e40f0462:::rbd_object_map.11064af7243d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896608+0000", 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.020033526999999999, 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020058096000000001, 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2445 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896608+0000", 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896608+0000", 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896607+0000", 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896614+0000", 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4069999999999998e-06 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896633+0000", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9204000000000001e-05 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896643+0000", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6360000000000006e-06 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.916459+0000", 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.019816639 2024-09-19T21:34:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.916567+0000", 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107221 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2506 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904099+0000", 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012542486, 2024-09-19T21:34:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012640027, 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2506 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904099+0000", 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904099+0000", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904099+0000", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904104+0000", 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1749999999999996e-06 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904113+0000", 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9539999999999993e-06 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904119+0000", 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1079999999999998e-06 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2446 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896663+0000", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019978965000000001, 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020129718000000001, 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2446 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896663+0000", 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896663+0000", 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896662+0000", 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896669+0000", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5799999999999997e-06 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896680+0000", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1206000000000001e-05 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896689+0000", 2024-09-19T21:34:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9420000000000001e-06 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2447 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896696+0000", 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019945728999999999, 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020149093, 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2447 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896696+0000", 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896696+0000", 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896696+0000", 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896700+0000", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8860000000000003e-06 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896710+0000", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5889999999999992e-06 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896730+0000", 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9772e-05 2024-09-19T21:34:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2508 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904206+0000", 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.01243641, 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01269211, 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2508 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904206+0000", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904206+0000", 2024-09-19T21:34:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904205+0000", 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904209+0000", 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3839999999999999e-06 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904217+0000", 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9689999999999994e-06 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904223+0000", 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8359999999999997e-06 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2448 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896739+0000", 2024-09-19T21:34:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019902842, 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020210187000000001, 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2448 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896739+0000", 2024-09-19T21:34:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896739+0000", 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896738+0000", 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896744+0000", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.942e-06 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896755+0000", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0891e-05 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896766+0000", 2024-09-19T21:34:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0346000000000001e-05 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2510 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904236+0000", 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012405512, 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012780883999999999, 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2510 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904236+0000", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904236+0000", 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904236+0000", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904240+0000", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904246+0000", 2024-09-19T21:34:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.195e-06 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904251+0000", 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0560000000000004e-06 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2449 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896775+0000", 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019866767, 2024-09-19T21:34:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020292555, 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2449 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896775+0000", 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896775+0000", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896775+0000", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896779+0000", 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.904e-06 2024-09-19T21:34:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896790+0000", 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0036e-05 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896798+0000", 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1009999999999992e-06 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2511 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904289+0000", 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012353236, 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012838001999999999, 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2511 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904289+0000", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904289+0000", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904287+0000", 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904297+0000", 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0183000000000001e-05 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904313+0000", 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6064000000000001e-05 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904324+0000", 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1043000000000001e-05 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2450 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896805+0000", 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019837166, 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020377449999999998, 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2450 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896805+0000", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896805+0000", 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896804+0000", 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896809+0000", 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9420000000000003e-06 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896822+0000", 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2867999999999999e-05 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896831+0000", 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6759999999999996e-06 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2513 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904403+0000", 2024-09-19T21:34:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012239335, 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012832962, 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2513 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904403+0000", 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904403+0000", 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904401+0000", 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904412+0000", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0061000000000001e-05 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904424+0000", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2598e-05 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904434+0000", 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8039999999999996e-06 2024-09-19T21:34:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2451 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896838+0000", 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019803594000000001, 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020448144000000001, 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2451 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896838+0000", 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896838+0000", 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896838+0000", 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896842+0000", 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7600000000000002e-06 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896853+0000", 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0823e-05 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896861+0000", 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2452 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896868+0000", 2024-09-19T21:34:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019773753000000002, 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020468884, 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2452 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896868+0000", 2024-09-19T21:34:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896868+0000", 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896868+0000", 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896889+0000", 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1072000000000001e-05 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896900+0000", 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1430999999999999e-05 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896909+0000", 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9959999999999999e-06 2024-09-19T21:34:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2515 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904498+0000", 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012143502, 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012889668, 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2515 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904498+0000", 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904498+0000", 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904498+0000", 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904503+0000", 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.135e-06 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904513+0000", 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0283e-05 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904521+0000", 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8509999999999999e-06 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2453 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896918+0000", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019724274, 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020520668999999998, 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2453 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896918+0000", 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896918+0000", 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896917+0000", 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896925+0000", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3699999999999995e-06 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896935+0000", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0195e-05 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896943+0000", 2024-09-19T21:34:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0220000000000004e-06 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2517 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904541+0000", 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012100947000000001, 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012952685, 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2517 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904541+0000", 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904541+0000", 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904540+0000", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904546+0000", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9270000000000002e-06 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904566+0000", 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0214e-05 2024-09-19T21:34:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904575+0000", 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2390000000000002e-06 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2454 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896950+0000", 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019691531000000002, 2024-09-19T21:34:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020593935000000001, 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2454 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896950+0000", 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896950+0000", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896950+0000", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896955+0000", 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7280000000000003e-06 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896964+0000", 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7280000000000007e-06 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896972+0000", 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0499999999999992e-06 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2518 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904587+0000", 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012054979, 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01300834, 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2518 2024-09-19T21:34:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904587+0000", 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904587+0000", 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904586+0000", 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904592+0000", 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.541e-06 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904601+0000", 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3619999999999995e-06 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904608+0000", 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.985e-06 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2455 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896979+0000", 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019662555000000002, 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020671855999999999, 2024-09-19T21:34:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2455 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896979+0000", 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896979+0000", 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896979+0000", 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896984+0000", 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7249999999999997e-06 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896993+0000", 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9219999999999992e-06 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897002+0000", 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3000000000000002e-06 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2456 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897009+0000", 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019632655999999998, 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020692980999999999, 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2456 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897009+0000", 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897009+0000", 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897009+0000", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897014+0000", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9479999999999999e-06 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897024+0000", 2024-09-19T21:34:40.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0271999999999999e-05 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897032+0000", 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9310000000000008e-06 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2520 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904684+0000", 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.011958228, 2024-09-19T21:34:40.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013053436999999999, 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2520 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904684+0000", 2024-09-19T21:34:40.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904684+0000", 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904683+0000", 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904688+0000", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.028e-06 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904698+0000", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6250000000000002e-06 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904705+0000", 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3549999999999999e-06 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2465 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897120+0000", 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019521698000000001, 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020643137999999998, 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2465 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897120+0000", 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897120+0000", 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897119+0000", 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897128+0000", 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.799e-06 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897142+0000", 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3954999999999999e-05 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897152+0000", 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7769999999999997e-06 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2521 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904723+0000", 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.011918544999999999, 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013068623999999999, 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2521 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904723+0000", 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904723+0000", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904723+0000", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904731+0000", 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1699999999999997e-06 2024-09-19T21:34:41.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904741+0000", 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0441e-05 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904749+0000", 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.362e-06 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2466 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897174+0000", 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019467862999999998, 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020644356999999999, 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2466 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897174+0000", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897174+0000", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897173+0000", 2024-09-19T21:34:41.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897179+0000", 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6720000000000001e-06 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897188+0000", 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1169999999999993e-06 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897195+0000", 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4069999999999998e-06 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2523 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904760+0000", 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.011881753, 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013085304000000001, 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2523 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904760+0000", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904760+0000", 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904760+0000", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904764+0000", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7570000000000004e-06 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904785+0000", 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0516000000000001e-05 2024-09-19T21:34:41.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904812+0000", 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6650000000000001e-05 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2467 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897210+0000", 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019431865, 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02066103, 2024-09-19T21:34:41.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2467 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897210+0000", 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897210+0000", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897209+0000", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897214+0000", 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.921e-06 2024-09-19T21:34:41.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897223+0000", 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0769999999999997e-06 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897230+0000", 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3960000000000003e-06 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2525 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904833+0000", 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.011808686000000001, 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013071187, 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2525 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904833+0000", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904833+0000", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904832+0000", 2024-09-19T21:34:41.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904838+0000", 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2179999999999996e-06 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904849+0000", 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0504e-05 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904858+0000", 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8340000000000005e-06 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2468 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897276+0000", 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.01936589, 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020664993, 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2468 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897276+0000", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897276+0000", 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897275+0000", 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897281+0000", 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2099999999999998e-06 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897292+0000", 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0216e-05 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897299+0000", 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4399999999999999e-06 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2526 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904865+0000", 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.011777123, 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013126232999999999, 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2526 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904865+0000", 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904865+0000", 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904864+0000", 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904869+0000", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904895+0000", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5698999999999999e-05 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904905+0000", 2024-09-19T21:34:41.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9129999999999998e-06 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2469 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897309+0000", 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019332766000000001, 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020732295000000001, 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2469 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897309+0000", 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897309+0000", 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897309+0000", 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897313+0000", 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2100000000000003e-06 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897320+0000", 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4240000000000004e-06 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897326+0000", 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6160000000000001e-06 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2470 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897380+0000", 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019261484999999998, 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02070512, 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2470 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897380+0000", 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897380+0000", 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897380+0000", 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897386+0000", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2469999999999997e-06 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897436+0000", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9709999999999997e-05 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897444+0000", 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0669999999999992e-06 2024-09-19T21:34:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2471 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897455+0000", 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019187238999999998, 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020674221, 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2471 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897455+0000", 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897455+0000", 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897454+0000", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897459+0000", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7450000000000003e-06 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897483+0000", 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4026999999999999e-05 2024-09-19T21:34:41.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897490+0000", 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7610000000000002e-06 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2472 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897507+0000", 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019135096000000001, 2024-09-19T21:34:41.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020667826, 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2472 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897507+0000", 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897507+0000", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897506+0000", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897511+0000", 2024-09-19T21:34:41.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9869999999999999e-06 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897521+0000", 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0266e-05 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897527+0000", 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1310000000000003e-06 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2473 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.897537+0000", 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.019105104000000001, 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.020689164999999999, 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2473 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897537+0000", 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897537+0000", 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897536+0000", 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897540+0000", 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000003e-06 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897548+0000", 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5399999999999998e-06 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.897560+0000", 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2305e-05 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2494 3.7 3.f1b76967 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.039 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.903711+0000", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012930468000000001, 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014568696000000001, 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2494 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903711+0000", 2024-09-19T21:34:41.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903711+0000", 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903710+0000", 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903725+0000", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5277999999999998e-05 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903743+0000", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7391e-05 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903754+0000", 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.166e-05 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2496 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.903809+0000", 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012832512000000001, 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014529380999999999, 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2496 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903809+0000", 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903809+0000", 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903809+0000", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903821+0000", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1817000000000001e-05 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903830+0000", 2024-09-19T21:34:41.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.098e-06 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903836+0000", 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5230000000000001e-06 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2498 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.903852+0000", 2024-09-19T21:34:41.046 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.01278966, 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014538034999999999, 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2498 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903852+0000", 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903852+0000", 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903852+0000", 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903856+0000", 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7960000000000003e-06 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903864+0000", 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903869+0000", 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3600000000000004e-06 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2500 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.903937+0000", 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012705395, 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01450509, 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2500 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903937+0000", 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903937+0000", 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903936+0000", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903940+0000", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4709999999999999e-06 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903946+0000", 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9170000000000003e-06 2024-09-19T21:34:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.903950+0000", 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2443 3.7 3:e40f0462:::rbd_object_map.11064af7243d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896493+0000", 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.020148816999999999, 2024-09-19T21:34:41.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.021991143000000001, 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2443 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896493+0000", 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896493+0000", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896492+0000", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896500+0000", 2024-09-19T21:34:41.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4619999999999999e-06 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896513+0000", 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2982e-05 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896522+0000", 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3670000000000003e-06 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.914461+0000", 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.017938662000000001 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.914509+0000", 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8735e-05 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 3", 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.914723+0000", 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021390299999999999 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.916418+0000", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016948530000000001 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.918416+0000", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001997511 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.918458+0000", 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2205e-05 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.918484+0000", 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6489e-05 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2502 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904017+0000", 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012624663, 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014563052, 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2502 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904017+0000", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904017+0000", 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904016+0000", 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904022+0000", 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5040000000000002e-06 2024-09-19T21:34:41.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904031+0000", 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2790000000000004e-06 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904039+0000", 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0760000000000003e-06 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2444 3.7 3:e40f0462:::rbd_object_map.11064af7243d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.896536+0000", 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.02010603, 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022090209999999999, 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for sub ops", 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2444 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.063 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896536+0000", 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896536+0000", 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896535+0000", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896554+0000", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8958e-05 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896566+0000", 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2248e-05 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.896585+0000", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9099000000000001e-05 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.915554+0000", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.018968450000000001 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.915650+0000", 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6125000000000005e-05 2024-09-19T21:34:41.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 3", 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.915937+0000", 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028715499999999998 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.918396+0000", 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0024588420000000001 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2504 3.7 3.4620f027 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.904048+0000", 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.012593797, 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014621471, 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2504 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904048+0000", 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904048+0000", 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904047+0000", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904052+0000", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.216e-06 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904059+0000", 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6159999999999996e-06 2024-09-19T21:34:41.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.904065+0000", 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1900000000000003e-06 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 42 2024-09-19T21:34:41.071 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:41.073 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:41.070+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:41.083 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.5 to 1 (10000) 2024-09-19T21:34:41.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:34:41.096 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-19T21:34:41.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:41.173 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:41.170+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:41.220 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone2 from 0xdf6a000 (overlap 0x114e600) to 0x0 2024-09-19T21:34:41.245 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:41.245 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:41.245 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:41.245 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:41.245 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:41.304 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:41.302+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:41.374 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:41.370+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:41.384 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:41.384 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:41.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9815989109999999, 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9815786040000001, 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:34:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9815591420000001, 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:34:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:34:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:34:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9815248309999998, 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9814993310000002, 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9775461019999998, 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:34:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9775089779999999, 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:34:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9774717590000002, 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.97742737, 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:34:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:34:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9774096829999999, 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:34:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9773847330000001, 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:34:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9773454419999998, 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.977308039, 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.97729021, 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:34:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.973641432, 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.973616888, 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9735982729999999, 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:34:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9735488669999999, 2024-09-19T21:34:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:34:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:34:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.973530469, 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:34:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:34:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9735046999999999, 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:34:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:34:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.472 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:41.475 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:41.470+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:41.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:41.501 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:41.520 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-19T21:34:41.084008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8618751,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5970,"num_read_kb":9273,"num_write":810,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":675,"ondisk_log_size":675,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":186240,"kb_used_data":3620,"kb_used_omap":9,"kb_used_meta":182518,"kb_avail":566044800,"statfs":{"total":579820584960,"available":579629875200,"internally_reserved":0,"allocated":3706880,"data_stored":2423175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":186899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7572045,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":5758,"num_read_kb":9049,"num_write":631,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039812"},"pg_stats":[{"pgid":"3.1f","version":"30'23","reported_seq":115,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.139937+0000","last_change":"2024-09-19T21:34:32.012011+0000","last_active":"2024-09-19T21:34:38.139937+0000","last_peered":"2024-09-19T21:34:38.139937+0000","last_clean":"2024-09-19T21:34:38.139937+0000","last_became_active":"2024-09-19T21:34:30.013688+0000","last_became_peered":"2024-09-19T21:34:30.013688+0000","last_unstale":"2024-09-19T21:34:38.139937+0000","last_undegraded":"2024-09-19T21:34:38.139937+0000","last_fullsized":"2024-09-19T21:34:38.139937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:27:13.050028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":434,"num_write":23,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023693+0000","last_change":"2024-09-19T21:34:32.006487+0000","last_active":"2024-09-19T21:34:33.023693+0000","last_peered":"2024-09-19T21:34:33.023693+0000","last_clean":"2024-09-19T21:34:33.023693+0000","last_became_active":"2024-09-19T21:34:30.011721+0000","last_became_peered":"2024-09-19T21:34:30.011721+0000","last_unstale":"2024-09-19T21:34:33.023693+0000","last_undegraded":"2024-09-19T21:34:33.023693+0000","last_fullsized":"2024-09-19T21:34:33.023693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:21.923196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"30'8","reported_seq":45,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055230+0000","last_change":"2024-09-19T21:34:32.024269+0000","last_active":"2024-09-19T21:34:38.055230+0000","last_peered":"2024-09-19T21:34:38.055230+0000","last_clean":"2024-09-19T21:34:38.055230+0000","last_became_active":"2024-09-19T21:34:30.010774+0000","last_became_peered":"2024-09-19T21:34:30.010774+0000","last_unstale":"2024-09-19T21:34:38.055230+0000","last_undegraded":"2024-09-19T21:34:38.055230+0000","last_fullsized":"2024-09-19T21:34:38.055230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:59.647828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":272,"num_write":8,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"30'90","reported_seq":1477,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.217967+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:34:38.217967+0000","last_peered":"2024-09-19T21:34:38.217967+0000","last_clean":"2024-09-19T21:34:38.217967+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:34:38.217967+0000","last_undegraded":"2024-09-19T21:34:38.217967+0000","last_fullsized":"2024-09-19T21:34:38.217967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1713,"num_read_kb":1480,"num_write":55,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"28'1","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.030643+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:34:36.030643+0000","last_peered":"2024-09-19T21:34:36.030643+0000","last_clean":"2024-09-19T21:34:36.030643+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:34:36.030643+0000","last_undegraded":"2024-09-19T21:34:36.030643+0000","last_fullsized":"2024-09-19T21:34:36.030643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":456192,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"29'13","reported_seq":219,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.216408+0000","last_change":"2024-09-19T21:34:32.001087+0000","last_active":"2024-09-19T21:34:38.216408+0000","last_peered":"2024-09-19T21:34:38.216408+0000","last_clean":"2024-09-19T21:34:38.216408+0000","last_became_active":"2024-09-19T21:34:30.002669+0000","last_became_peered":"2024-09-19T21:34:30.002669+0000","last_unstale":"2024-09-19T21:34:38.216408+0000","last_undegraded":"2024-09-19T21:34:38.216408+0000","last_fullsized":"2024-09-19T21:34:38.216408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:54:28.569315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":187,"num_read_kb":346,"num_write":13,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048921+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:34:37.048921+0000","last_peered":"2024-09-19T21:34:37.048921+0000","last_clean":"2024-09-19T21:34:37.048921+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:34:37.048921+0000","last_undegraded":"2024-09-19T21:34:37.048921+0000","last_fullsized":"2024-09-19T21:34:37.048921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"28'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.405037+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:34:36.405037+0000","last_peered":"2024-09-19T21:34:36.405037+0000","last_clean":"2024-09-19T21:34:36.405037+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:34:36.405037+0000","last_undegraded":"2024-09-19T21:34:36.405037+0000","last_fullsized":"2024-09-19T21:34:36.405037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":3,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"30'8","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.139995+0000","last_change":"2024-09-19T21:34:32.012032+0000","last_active":"2024-09-19T21:34:38.139995+0000","last_peered":"2024-09-19T21:34:38.139995+0000","last_clean":"2024-09-19T21:34:38.139995+0000","last_became_active":"2024-09-19T21:34:30.011066+0000","last_became_peered":"2024-09-19T21:34:30.011066+0000","last_unstale":"2024-09-19T21:34:38.139995+0000","last_undegraded":"2024-09-19T21:34:38.139995+0000","last_fullsized":"2024-09-19T21:34:38.139995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:40:19.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":20,"num_write":8,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.042199+0000","last_change":"2024-09-19T21:34:32.024453+0000","last_active":"2024-09-19T21:34:37.042199+0000","last_peered":"2024-09-19T21:34:37.042199+0000","last_clean":"2024-09-19T21:34:37.042199+0000","last_became_active":"2024-09-19T21:34:30.006875+0000","last_became_peered":"2024-09-19T21:34:30.006875+0000","last_unstale":"2024-09-19T21:34:37.042199+0000","last_undegraded":"2024-09-19T21:34:37.042199+0000","last_fullsized":"2024-09-19T21:34:37.042199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:26:13.507158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"30'8","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.155227+0000","last_change":"2024-09-19T21:34:32.006201+0000","last_active":"2024-09-19T21:34:38.155227+0000","last_peered":"2024-09-19T21:34:38.155227+0000","last_clean":"2024-09-19T21:34:38.155227+0000","last_became_active":"2024-09-19T21:34:30.008159+0000","last_became_peered":"2024-09-19T21:34:30.008159+0000","last_unstale":"2024-09-19T21:34:38.155227+0000","last_undegraded":"2024-09-19T21:34:38.155227+0000","last_fullsized":"2024-09-19T21:34:38.155227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:44:57.994848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"29'3","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.225776+0000","last_change":"2024-09-19T21:34:32.002842+0000","last_active":"2024-09-19T21:34:38.225776+0000","last_peered":"2024-09-19T21:34:38.225776+0000","last_clean":"2024-09-19T21:34:38.225776+0000","last_became_active":"2024-09-19T21:34:30.010116+0000","last_became_peered":"2024-09-19T21:34:30.010116+0000","last_unstale":"2024-09-19T21:34:38.225776+0000","last_undegraded":"2024-09-19T21:34:38.225776+0000","last_fullsized":"2024-09-19T21:34:38.225776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:38:12.596445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":446464,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":279,"num_read_kb":768,"num_write":3,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.040063+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:37.040063+0000","last_peered":"2024-09-19T21:34:37.040063+0000","last_clean":"2024-09-19T21:34:37.040063+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:37.040063+0000","last_undegraded":"2024-09-19T21:34:37.040063+0000","last_fullsized":"2024-09-19T21:34:37.040063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'17","reported_seq":50,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.156818+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:34:38.156818+0000","last_peered":"2024-09-19T21:34:38.156818+0000","last_clean":"2024-09-19T21:34:38.156818+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:34:38.156818+0000","last_undegraded":"2024-09-19T21:34:38.156818+0000","last_fullsized":"2024-09-19T21:34:38.156818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":161,"num_write":17,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048845+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:37.048845+0000","last_peered":"2024-09-19T21:34:37.048845+0000","last_clean":"2024-09-19T21:34:37.048845+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:37.048845+0000","last_undegraded":"2024-09-19T21:34:37.048845+0000","last_fullsized":"2024-09-19T21:34:37.048845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'9","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.161294+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:34:38.161294+0000","last_peered":"2024-09-19T21:34:38.161294+0000","last_clean":"2024-09-19T21:34:38.161294+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:34:38.161294+0000","last_undegraded":"2024-09-19T21:34:38.161294+0000","last_fullsized":"2024-09-19T21:34:38.161294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":6144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039786+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:37.039786+0000","last_peered":"2024-09-19T21:34:37.039786+0000","last_clean":"2024-09-19T21:34:37.039786+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:37.039786+0000","last_undegraded":"2024-09-19T21:34:37.039786+0000","last_fullsized":"2024-09-19T21:34:37.039786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'11","reported_seq":46,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.161226+0000","last_change":"2024-09-19T21:34:32.024327+0000","last_active":"2024-09-19T21:34:38.161226+0000","last_peered":"2024-09-19T21:34:38.161226+0000","last_clean":"2024-09-19T21:34:38.161226+0000","last_became_active":"2024-09-19T21:34:30.007041+0000","last_became_peered":"2024-09-19T21:34:30.007041+0000","last_unstale":"2024-09-19T21:34:38.161226+0000","last_undegraded":"2024-09-19T21:34:38.161226+0000","last_fullsized":"2024-09-19T21:34:38.161226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:31:26.974439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":40960,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":204,"num_write":11,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023253+0000","last_change":"2024-09-19T21:34:31.005184+0000","last_active":"2024-09-19T21:34:33.023253+0000","last_peered":"2024-09-19T21:34:33.023253+0000","last_clean":"2024-09-19T21:34:33.023253+0000","last_became_active":"2024-09-19T21:34:31.004633+0000","last_became_peered":"2024-09-19T21:34:31.004633+0000","last_unstale":"2024-09-19T21:34:33.023253+0000","last_undegraded":"2024-09-19T21:34:33.023253+0000","last_fullsized":"2024-09-19T21:34:33.023253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:21.400292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"30'4","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055278+0000","last_change":"2024-09-19T21:34:32.024762+0000","last_active":"2024-09-19T21:34:38.055278+0000","last_peered":"2024-09-19T21:34:38.055278+0000","last_clean":"2024-09-19T21:34:38.055278+0000","last_became_active":"2024-09-19T21:34:30.006903+0000","last_became_peered":"2024-09-19T21:34:30.006903+0000","last_unstale":"2024-09-19T21:34:38.055278+0000","last_undegraded":"2024-09-19T21:34:38.055278+0000","last_fullsized":"2024-09-19T21:34:38.055278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.226419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"30'54","reported_seq":75,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.154724+0000","last_change":"2024-09-19T21:34:32.006802+0000","last_active":"2024-09-19T21:34:38.154724+0000","last_peered":"2024-09-19T21:34:38.154724+0000","last_clean":"2024-09-19T21:34:38.154724+0000","last_became_active":"2024-09-19T21:34:30.011547+0000","last_became_peered":"2024-09-19T21:34:30.011547+0000","last_unstale":"2024-09-19T21:34:38.154724+0000","last_undegraded":"2024-09-19T21:34:38.154724+0000","last_fullsized":"2024-09-19T21:34:38.154724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:45:47.281981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":182,"num_write":83,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.045103+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:37.045103+0000","last_peered":"2024-09-19T21:34:37.045103+0000","last_clean":"2024-09-19T21:34:37.045103+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:37.045103+0000","last_undegraded":"2024-09-19T21:34:37.045103+0000","last_fullsized":"2024-09-19T21:34:37.045103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'10","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.154539+0000","last_change":"2024-09-19T21:34:32.011746+0000","last_active":"2024-09-19T21:34:38.154539+0000","last_peered":"2024-09-19T21:34:38.154539+0000","last_clean":"2024-09-19T21:34:38.154539+0000","last_became_active":"2024-09-19T21:34:30.013347+0000","last_became_peered":"2024-09-19T21:34:30.013347+0000","last_unstale":"2024-09-19T21:34:38.154539+0000","last_undegraded":"2024-09-19T21:34:38.154539+0000","last_fullsized":"2024-09-19T21:34:38.154539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.129809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":16,"num_write":10,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023328+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:33.023328+0000","last_peered":"2024-09-19T21:34:33.023328+0000","last_clean":"2024-09-19T21:34:33.023328+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:33.023328+0000","last_undegraded":"2024-09-19T21:34:33.023328+0000","last_fullsized":"2024-09-19T21:34:33.023328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"28'53","reported_seq":632,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048646+0000","last_change":"2024-09-19T21:34:32.024303+0000","last_active":"2024-09-19T21:34:37.048646+0000","last_peered":"2024-09-19T21:34:37.048646+0000","last_clean":"2024-09-19T21:34:37.048646+0000","last_became_active":"2024-09-19T21:34:30.009730+0000","last_became_peered":"2024-09-19T21:34:30.009730+0000","last_unstale":"2024-09-19T21:34:37.048646+0000","last_undegraded":"2024-09-19T21:34:37.048646+0000","last_fullsized":"2024-09-19T21:34:37.048646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1167,"num_read_kb":1002,"num_write":28,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.041987+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:37.041987+0000","last_peered":"2024-09-19T21:34:37.041987+0000","last_clean":"2024-09-19T21:34:37.041987+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:37.041987+0000","last_undegraded":"2024-09-19T21:34:37.041987+0000","last_fullsized":"2024-09-19T21:34:37.041987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"28'8","reported_seq":43,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.714367+0000","last_change":"2024-09-19T21:34:32.005130+0000","last_active":"2024-09-19T21:34:36.714367+0000","last_peered":"2024-09-19T21:34:36.714367+0000","last_clean":"2024-09-19T21:34:36.714367+0000","last_became_active":"2024-09-19T21:34:30.009669+0000","last_became_peered":"2024-09-19T21:34:30.009669+0000","last_unstale":"2024-09-19T21:34:36.714367+0000","last_undegraded":"2024-09-19T21:34:36.714367+0000","last_fullsized":"2024-09-19T21:34:36.714367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:20:42.350074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":69,"num_write":8,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039552+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:37.039552+0000","last_peered":"2024-09-19T21:34:37.039552+0000","last_clean":"2024-09-19T21:34:37.039552+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:37.039552+0000","last_undegraded":"2024-09-19T21:34:37.039552+0000","last_fullsized":"2024-09-19T21:34:37.039552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039634+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:37.039634+0000","last_peered":"2024-09-19T21:34:37.039634+0000","last_clean":"2024-09-19T21:34:37.039634+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:37.039634+0000","last_undegraded":"2024-09-19T21:34:37.039634+0000","last_fullsized":"2024-09-19T21:34:37.039634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"30'5","reported_seq":1073,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.153697+0000","last_change":"2024-09-19T21:34:32.024996+0000","last_active":"2024-09-19T21:34:38.153697+0000","last_peered":"2024-09-19T21:34:38.153697+0000","last_clean":"2024-09-19T21:34:38.153697+0000","last_became_active":"2024-09-19T21:34:30.009636+0000","last_became_peered":"2024-09-19T21:34:30.009636+0000","last_unstale":"2024-09-19T21:34:38.153697+0000","last_undegraded":"2024-09-19T21:34:38.153697+0000","last_fullsized":"2024-09-19T21:34:38.153697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:11.666768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":2233344,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1050,"num_read_kb":2281,"num_write":5,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"29'6","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.947870+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:34:37.947870+0000","last_peered":"2024-09-19T21:34:37.947870+0000","last_clean":"2024-09-19T21:34:37.947870+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:34:37.947870+0000","last_undegraded":"2024-09-19T21:34:37.947870+0000","last_fullsized":"2024-09-19T21:34:37.947870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"30'17","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.160187+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:34:38.160187+0000","last_peered":"2024-09-19T21:34:38.160187+0000","last_clean":"2024-09-19T21:34:38.160187+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:34:38.160187+0000","last_undegraded":"2024-09-19T21:34:38.160187+0000","last_fullsized":"2024-09-19T21:34:38.160187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":20,"num_write":17,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023176+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:34:33.023176+0000","last_peered":"2024-09-19T21:34:33.023176+0000","last_clean":"2024-09-19T21:34:33.023176+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:34:33.023176+0000","last_undegraded":"2024-09-19T21:34:33.023176+0000","last_fullsized":"2024-09-19T21:34:33.023176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"27'4","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.045295+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:34:37.045295+0000","last_peered":"2024-09-19T21:34:37.045295+0000","last_clean":"2024-09-19T21:34:37.045295+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:34:37.045295+0000","last_undegraded":"2024-09-19T21:34:37.045295+0000","last_fullsized":"2024-09-19T21:34:37.045295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":1261056,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":45,"num_write":4,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"29'2","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.654360+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:34:37.654360+0000","last_peered":"2024-09-19T21:34:37.654360+0000","last_clean":"2024-09-19T21:34:37.654360+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:34:37.654360+0000","last_undegraded":"2024-09-19T21:34:37.654360+0000","last_fullsized":"2024-09-19T21:34:37.654360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"30'16","reported_seq":59,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.150967+0000","last_change":"2024-09-19T21:34:32.024576+0000","last_active":"2024-09-19T21:34:38.150967+0000","last_peered":"2024-09-19T21:34:38.150967+0000","last_clean":"2024-09-19T21:34:38.150967+0000","last_became_active":"2024-09-19T21:34:30.006185+0000","last_became_peered":"2024-09-19T21:34:30.006185+0000","last_unstale":"2024-09-19T21:34:38.150967+0000","last_undegraded":"2024-09-19T21:34:38.150967+0000","last_fullsized":"2024-09-19T21:34:38.150967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:08:25.609727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":223,"num_write":16,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"30'161","reported_seq":182,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.222148+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:34:38.222148+0000","last_peered":"2024-09-19T21:34:38.222148+0000","last_clean":"2024-09-19T21:34:38.222148+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:34:38.222148+0000","last_undegraded":"2024-09-19T21:34:38.222148+0000","last_fullsized":"2024-09-19T21:34:38.222148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":42557,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":848,"num_read_kb":692,"num_write":286,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"30'4","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.133090+0000","last_change":"2024-09-19T21:34:32.000772+0000","last_active":"2024-09-19T21:34:38.133090+0000","last_peered":"2024-09-19T21:34:38.133090+0000","last_clean":"2024-09-19T21:34:38.133090+0000","last_became_active":"2024-09-19T21:34:30.007945+0000","last_became_peered":"2024-09-19T21:34:30.007945+0000","last_unstale":"2024-09-19T21:34:38.133090+0000","last_undegraded":"2024-09-19T21:34:38.133090+0000","last_fullsized":"2024-09-19T21:34:38.133090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:20:07.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"30'8","reported_seq":55,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.054603+0000","last_change":"2024-09-19T21:34:32.023713+0000","last_active":"2024-09-19T21:34:38.054603+0000","last_peered":"2024-09-19T21:34:38.054603+0000","last_clean":"2024-09-19T21:34:38.054603+0000","last_became_active":"2024-09-19T21:34:30.003969+0000","last_became_peered":"2024-09-19T21:34:30.003969+0000","last_unstale":"2024-09-19T21:34:38.054603+0000","last_undegraded":"2024-09-19T21:34:38.054603+0000","last_fullsized":"2024-09-19T21:34:38.054603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:36.748615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":280,"num_write":8,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.055666+0000","last_change":"2024-09-19T21:34:32.011754+0000","last_active":"2024-09-19T21:34:37.055666+0000","last_peered":"2024-09-19T21:34:37.055666+0000","last_clean":"2024-09-19T21:34:37.055666+0000","last_became_active":"2024-09-19T21:34:30.012126+0000","last_became_peered":"2024-09-19T21:34:30.012126+0000","last_unstale":"2024-09-19T21:34:37.055666+0000","last_undegraded":"2024-09-19T21:34:37.055666+0000","last_fullsized":"2024-09-19T21:34:37.055666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:32.355336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"30'9","reported_seq":153,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055683+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:34:38.055683+0000","last_peered":"2024-09-19T21:34:38.055683+0000","last_clean":"2024-09-19T21:34:38.055683+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:34:38.055683+0000","last_undegraded":"2024-09-19T21:34:38.055683+0000","last_fullsized":"2024-09-19T21:34:38.055683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":3537408,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":674,"num_write":9,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8028364,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5854,"num_read_kb":9173,"num_write":649,"num_write_kb":2625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":267160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":555,"ondisk_log_size":555,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":946176,"data_stored":733999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":417792,"data_stored":205613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":974848,"data_stored":760509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":598016,"data_stored":362883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":12,"seq":51539607560,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":307200,"data_stored":115137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":74347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":30270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":23102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":49198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":16419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:41.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:41.576 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:41.569+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:41.591 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:41.591 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:41.591 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:41.591 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:41.677 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:41.673+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:41.713 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:41.713 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:41.713 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:41.713 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:41.713 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:41.777 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:41.772+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:104 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.362086073, 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077086730000000001, 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474560+0000", 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474577+0000", 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7283e-05 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474609+0000", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2316e-05 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474622+0000", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.305e-05 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474765+0000", 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014250999999999999 2024-09-19T21:34:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474825+0000", 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0529000000000003e-05 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482210+0000", 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073848660000000003 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482270+0000", 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9786e-05 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.360892869000001, 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006550213, 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475753+0000", 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475771+0000", 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8067000000000002e-05 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475808+0000", 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6665000000000002e-05 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475825+0000", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7827000000000001e-05 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475942+0000", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011702 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475984+0000", 2024-09-19T21:34:41.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1585999999999998e-05 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482279+0000", 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062948850000000001 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482305+0000", 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5786999999999999e-05 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.360700510000001, 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087792770000000003, 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:41.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475945+0000", 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476124+0000", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000178949 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476158+0000", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3200000000000001e-05 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476181+0000", 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3898999999999999e-05 2024-09-19T21:34:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476764+0000", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058244000000000004 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476831+0000", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6792999999999996e-05 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484666+0000", 2024-09-19T21:34:41.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078356450000000005 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484726+0000", 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9701000000000003e-05 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:41.854 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.359895721000001, 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080060370000000006, 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:41.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476750+0000", 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476768+0000", 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8566000000000001e-05 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476817+0000", 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8523000000000003e-05 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476833+0000", 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6079999999999999e-05 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477864+0000", 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001031243 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477897+0000", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3163000000000001e-05 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484733+0000", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068351740000000003 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484758+0000", 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5075e-05 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.858 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.359313437000001, 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007490137, 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477332+0000", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477346+0000", 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3219e-05 2024-09-19T21:34:41.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477374+0000", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8799999999999999e-05 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477388+0000", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3129000000000001e-05 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478476+0000", 2024-09-19T21:34:41.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010886030000000001 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478507+0000", 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1083e-05 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484770+0000", 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062626540000000003 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484824+0000", 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4151999999999999e-05 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.358664911999998, 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068625409999999998, 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477981+0000", 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478037+0000", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6742e-05 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478068+0000", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0303000000000001e-05 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478094+0000", 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5925999999999999e-05 2024-09-19T21:34:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479202+0000", 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011083219999999999 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479240+0000", 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7592999999999997e-05 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484826+0000", 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055858959999999999 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484845+0000", 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9425000000000001e-05 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 25.357671762999999, 2024-09-19T21:34:41.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066455869999999997, 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478974+0000", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478988+0000", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4202999999999999e-05 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479013+0000", 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978999999999999e-05 2024-09-19T21:34:41.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479036+0000", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2959000000000002e-05 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480591+0000", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015545730000000001 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480637+0000", 2024-09-19T21:34:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5939000000000002e-05 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485516+0000", 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004879899 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485621+0000", 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104715 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:621 3.1f 3:fe16a831:::rbd_data.1106b3d15a22.00000000000001a7:head [set-alloc-hint object_size 524288 write_size 524288,write 240640~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8093586760000004, 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067125730000000003, 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 621 2024-09-19T21:34:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027286+0000", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027302+0000", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5809999999999999e-05 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027339+0000", 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7036000000000003e-05 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027355+0000", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5265999999999999e-05 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027495+0000", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014067999999999999 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027888+0000", 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039285500000000001 2024-09-19T21:34:41.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.028329+0000", 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044053700000000001 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.030559+0000", 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022299149999999998 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033825+0000", 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032660319999999999 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033901+0000", 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6384999999999995e-05 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.034001+0000", 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100264 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2992 3.0 3:01a40bc1:::rbd_data.1106b3d15a22.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.6883149949999998, 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066149140000000004, 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2992 2024-09-19T21:34:41.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148331+0000", 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148340+0000", 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6619999999999994e-06 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148357+0000", 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7674000000000001e-05 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148371+0000", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3603e-05 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150715+0000", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023444820000000002 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150853+0000", 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000137575 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.151152+0000", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029933299999999999 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153280+0000", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021272610000000001 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154735+0000", 2024-09-19T21:34:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001455837 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154867+0000", 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181099999999999 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154947+0000", 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9961000000000002e-05 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.883 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:1106 3.4 e32/23 3:203a8a6a:::rbd_data.11064af7243d.0000000000000023:head v 32'15, mlcod=30'13)", 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.623461+0000", 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.2131859380000001, 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064755070000000001, 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623461+0000", 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623461+0000", 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623461+0000", 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623470+0000", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9230000000000008e-06 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623489+0000", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9664999999999999e-05 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623503+0000", 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4163e-05 2024-09-19T21:34:41.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623574+0000", 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0434999999999999e-05 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.623600+0000", 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6457000000000001e-05 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.629899+0000", 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062982250000000002 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.629937+0000", 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8334000000000003e-05 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2156 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write 137216~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.99193649699999997, 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068932300000000002, 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2156 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.889 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844708+0000", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844727+0000", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8975e-05 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844757+0000", 2024-09-19T21:34:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0071000000000001e-05 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844769+0000", 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1848000000000001e-05 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844825+0000", 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5488999999999997e-05 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844987+0000", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016212099999999999 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.845120+0000", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013307299999999999 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.846757+0000", 2024-09-19T21:34:41.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016375389999999999 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851484+0000", 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047265140000000002 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851555+0000", 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0994000000000005e-05 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851604+0000", 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9172999999999998e-05 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3027 3.1f 3:f973b0ba:::rbd_data.11064af7243d.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.60468227399999996, 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014783325, 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3027 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231962+0000", 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231977+0000", 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4780000000000001e-05 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232006+0000", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9094e-05 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232025+0000", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8771e-05 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232146+0000", 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120834 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232377+0000", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023080400000000001 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232837+0000", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046002400000000001 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.243127+0000", 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290509999999999 2024-09-19T21:34:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246633+0000", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035057460000000001 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246695+0000", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1860999999999996e-05 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246748+0000", 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3065 3.1f 3:fb335e39:::rbd_data.11064af7243d.0000000000000021:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.60359543800000004, 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014219874, 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3065 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233050+0000", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233063+0000", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2850999999999999e-05 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233086+0000", 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3125999999999998e-05 2024-09-19T21:34:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233106+0000", 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0273000000000001e-05 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236241+0000", 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031351590000000002 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236353+0000", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011176899999999999 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236627+0000", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027411599999999998 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246167+0000", 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095400050000000007 2024-09-19T21:34:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247178+0000", 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001010628 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247224+0000", 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6786000000000002e-05 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247272+0000", 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7264e-05 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3075 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.60352302000000002, 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015477372, 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3075 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233123+0000", 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233134+0000", 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.046e-05 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233162+0000", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8530999999999998e-05 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233177+0000", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4894e-05 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239745+0000", 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065679969999999999 2024-09-19T21:34:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239854+0000", 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108999 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.240035+0000", 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018081799999999999 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247400+0000", 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073650909999999998 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248505+0000", 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011052779999999999 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248555+0000", 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0175e-05 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248602+0000", 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6343e-05 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3073 3.10 e33/23 3:0fcc0951:::rbd_data.11064af7243d.000000000000009a:head v 33'31, mlcod=32'26)", 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.60174741099999995, 2024-09-19T21:34:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078372670000000002, 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234899+0000", 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234914+0000", 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4630999999999999e-05 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234926+0000", 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1901e-05 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234934+0000", 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4100000000000008e-06 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235023+0000", 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8639000000000003e-05 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235048+0000", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5230000000000001e-05 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242558+0000", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075100929999999998 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242737+0000", 2024-09-19T21:34:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3072 3.1a e33/23 3:5c4f536b:::rbd_data.11064af7243d.0000000000000099:head v 33'32, mlcod=32'26)", 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.60149833399999997, 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074658509999999999, 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235148+0000", 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235157+0000", 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2169999999999994e-06 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235167+0000", 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235176+0000", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1419999999999993e-06 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235252+0000", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6489999999999994e-05 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235277+0000", 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4624999999999999e-05 2024-09-19T21:34:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242578+0000", 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073010280000000002 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242615+0000", 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7039999999999998e-05 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "pg_update_log_missing(3.4 epoch 33/23 rep_tid 343 entries 33'27 (0'0) error 3:21918efa:::rbd_data.11064af7243d.0000000000000015:head by client.4554.0:3042 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 32'24)", 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.59829296799999998, 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074741240000000004, 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238360+0000", 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238379+0000", 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8966000000000001e-05 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238393+0000", 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3621000000000001e-05 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238587+0000", 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019394200000000001 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.245828+0000", 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072414319999999999 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3069 3.4 e33/23 3:203a8a6a:::rbd_data.11064af7243d.0000000000000023:head v 33'29, mlcod=32'24)", 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.59824685600000005, 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077039919999999998, 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238408+0000", 2024-09-19T21:34:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4250000000000001e-06 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238414+0000", 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8249999999999999e-06 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238419+0000", 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4290000000000003e-06 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238781+0000", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036266899999999998 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238806+0000", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4320000000000001e-05 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246061+0000", 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072551999999999998 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246104+0000", 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3495000000000003e-05 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:5036 3.12 3:4d11eb75:::rbd_data.11064af7243d.00000000000000f4:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.55501163499999995, 2024-09-19T21:34:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070879050000000002, 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 5036 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281634+0000", 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281642+0000", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6199999999999999e-06 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281658+0000", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6283999999999999e-05 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281666+0000", 2024-09-19T21:34:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9899999999999997e-06 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281710+0000", 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4520000000000001e-05 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281780+0000", 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9342000000000001e-05 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281891+0000", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111544 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.286723+0000", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048321079999999999 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288632+0000", 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019087959999999999 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288684+0000", 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080999999999998e-05 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288724+0000", 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9347000000000001e-05 2024-09-19T21:34:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:5049 3.1a e33/23 3:5b4ddbac:::rbd_data.11064af7243d.0000000000000000:head v 33'37, mlcod=33'32)", 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.54425643599999995, 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069007460000000001, 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292390+0000", 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292397+0000", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.294e-06 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292410+0000", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2921999999999999e-05 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292417+0000", 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2190000000000001e-06 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294086+0000", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016694780000000001 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294119+0000", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2636999999999999e-05 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299247+0000", 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051283570000000001 2024-09-19T21:34:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299292+0000", 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4517000000000001e-05 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:41.942 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:41.943 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-19T21:34:41.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:41.946 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:41.947 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:41.947 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-19T21:34:41.084008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8618751,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5970,"num_read_kb":9273,"num_write":810,"num_write_kb":5043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":675,"ondisk_log_size":675,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":186240,"kb_used_data":3620,"kb_used_omap":9,"kb_used_meta":182518,"kb_avail":566044800,"statfs":{"total":579820584960,"available":579629875200,"internally_reserved":0,"allocated":3706880,"data_stored":2423175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":186899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7572045,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":5758,"num_read_kb":9049,"num_write":631,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039812"},"pg_stats":[{"pgid":"3.1f","version":"30'23","reported_seq":115,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.139937+0000","last_change":"2024-09-19T21:34:32.012011+0000","last_active":"2024-09-19T21:34:38.139937+0000","last_peered":"2024-09-19T21:34:38.139937+0000","last_clean":"2024-09-19T21:34:38.139937+0000","last_became_active":"2024-09-19T21:34:30.013688+0000","last_became_peered":"2024-09-19T21:34:30.013688+0000","last_unstale":"2024-09-19T21:34:38.139937+0000","last_undegraded":"2024-09-19T21:34:38.139937+0000","last_fullsized":"2024-09-19T21:34:38.139937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:27:13.050028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":434,"num_write":23,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023693+0000","last_change":"2024-09-19T21:34:32.006487+0000","last_active":"2024-09-19T21:34:33.023693+0000","last_peered":"2024-09-19T21:34:33.023693+0000","last_clean":"2024-09-19T21:34:33.023693+0000","last_became_active":"2024-09-19T21:34:30.011721+0000","last_became_peered":"2024-09-19T21:34:30.011721+0000","last_unstale":"2024-09-19T21:34:33.023693+0000","last_undegraded":"2024-09-19T21:34:33.023693+0000","last_fullsized":"2024-09-19T21:34:33.023693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:21.923196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"30'8","reported_seq":45,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055230+0000","last_change":"2024-09-19T21:34:32.024269+0000","last_active":"2024-09-19T21:34:38.055230+0000","last_peered":"2024-09-19T21:34:38.055230+0000","last_clean":"2024-09-19T21:34:38.055230+0000","last_became_active":"2024-09-19T21:34:30.010774+0000","last_became_peered":"2024-09-19T21:34:30.010774+0000","last_unstale":"2024-09-19T21:34:38.055230+0000","last_undegraded":"2024-09-19T21:34:38.055230+0000","last_fullsized":"2024-09-19T21:34:38.055230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:59.647828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":272,"num_write":8,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"30'90","reported_seq":1477,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.217967+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:34:38.217967+0000","last_peered":"2024-09-19T21:34:38.217967+0000","last_clean":"2024-09-19T21:34:38.217967+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:34:38.217967+0000","last_undegraded":"2024-09-19T21:34:38.217967+0000","last_fullsized":"2024-09-19T21:34:38.217967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1713,"num_read_kb":1480,"num_write":55,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"28'1","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.030643+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:34:36.030643+0000","last_peered":"2024-09-19T21:34:36.030643+0000","last_clean":"2024-09-19T21:34:36.030643+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:34:36.030643+0000","last_undegraded":"2024-09-19T21:34:36.030643+0000","last_fullsized":"2024-09-19T21:34:36.030643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":456192,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"29'13","reported_seq":219,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.216408+0000","last_change":"2024-09-19T21:34:32.001087+0000","last_active":"2024-09-19T21:34:38.216408+0000","last_peered":"2024-09-19T21:34:38.216408+0000","last_clean":"2024-09-19T21:34:38.216408+0000","last_became_active":"2024-09-19T21:34:30.002669+0000","last_became_peered":"2024-09-19T21:34:30.002669+0000","last_unstale":"2024-09-19T21:34:38.216408+0000","last_undegraded":"2024-09-19T21:34:38.216408+0000","last_fullsized":"2024-09-19T21:34:38.216408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:54:28.569315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":187,"num_read_kb":346,"num_write":13,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048921+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:34:37.048921+0000","last_peered":"2024-09-19T21:34:37.048921+0000","last_clean":"2024-09-19T21:34:37.048921+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:34:37.048921+0000","last_undegraded":"2024-09-19T21:34:37.048921+0000","last_fullsized":"2024-09-19T21:34:37.048921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"28'3","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.405037+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:34:36.405037+0000","last_peered":"2024-09-19T21:34:36.405037+0000","last_clean":"2024-09-19T21:34:36.405037+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:34:36.405037+0000","last_undegraded":"2024-09-19T21:34:36.405037+0000","last_fullsized":"2024-09-19T21:34:36.405037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":3,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"30'8","reported_seq":32,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.139995+0000","last_change":"2024-09-19T21:34:32.012032+0000","last_active":"2024-09-19T21:34:38.139995+0000","last_peered":"2024-09-19T21:34:38.139995+0000","last_clean":"2024-09-19T21:34:38.139995+0000","last_became_active":"2024-09-19T21:34:30.011066+0000","last_became_peered":"2024-09-19T21:34:30.011066+0000","last_unstale":"2024-09-19T21:34:38.139995+0000","last_undegraded":"2024-09-19T21:34:38.139995+0000","last_fullsized":"2024-09-19T21:34:38.139995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:40:19.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":20,"num_write":8,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.042199+0000","last_change":"2024-09-19T21:34:32.024453+0000","last_active":"2024-09-19T21:34:37.042199+0000","last_peered":"2024-09-19T21:34:37.042199+0000","last_clean":"2024-09-19T21:34:37.042199+0000","last_became_active":"2024-09-19T21:34:30.006875+0000","last_became_peered":"2024-09-19T21:34:30.006875+0000","last_unstale":"2024-09-19T21:34:37.042199+0000","last_undegraded":"2024-09-19T21:34:37.042199+0000","last_fullsized":"2024-09-19T21:34:37.042199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:26:13.507158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"30'8","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.155227+0000","last_change":"2024-09-19T21:34:32.006201+0000","last_active":"2024-09-19T21:34:38.155227+0000","last_peered":"2024-09-19T21:34:38.155227+0000","last_clean":"2024-09-19T21:34:38.155227+0000","last_became_active":"2024-09-19T21:34:30.008159+0000","last_became_peered":"2024-09-19T21:34:30.008159+0000","last_unstale":"2024-09-19T21:34:38.155227+0000","last_undegraded":"2024-09-19T21:34:38.155227+0000","last_fullsized":"2024-09-19T21:34:38.155227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:44:57.994848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"29'3","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.225776+0000","last_change":"2024-09-19T21:34:32.002842+0000","last_active":"2024-09-19T21:34:38.225776+0000","last_peered":"2024-09-19T21:34:38.225776+0000","last_clean":"2024-09-19T21:34:38.225776+0000","last_became_active":"2024-09-19T21:34:30.010116+0000","last_became_peered":"2024-09-19T21:34:30.010116+0000","last_unstale":"2024-09-19T21:34:38.225776+0000","last_undegraded":"2024-09-19T21:34:38.225776+0000","last_fullsized":"2024-09-19T21:34:38.225776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:38:12.596445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":446464,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":279,"num_read_kb":768,"num_write":3,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.040063+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:37.040063+0000","last_peered":"2024-09-19T21:34:37.040063+0000","last_clean":"2024-09-19T21:34:37.040063+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:37.040063+0000","last_undegraded":"2024-09-19T21:34:37.040063+0000","last_fullsized":"2024-09-19T21:34:37.040063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'17","reported_seq":50,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.156818+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:34:38.156818+0000","last_peered":"2024-09-19T21:34:38.156818+0000","last_clean":"2024-09-19T21:34:38.156818+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:34:38.156818+0000","last_undegraded":"2024-09-19T21:34:38.156818+0000","last_fullsized":"2024-09-19T21:34:38.156818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":161,"num_write":17,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048845+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:37.048845+0000","last_peered":"2024-09-19T21:34:37.048845+0000","last_clean":"2024-09-19T21:34:37.048845+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:37.048845+0000","last_undegraded":"2024-09-19T21:34:37.048845+0000","last_fullsized":"2024-09-19T21:34:37.048845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'9","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.161294+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:34:38.161294+0000","last_peered":"2024-09-19T21:34:38.161294+0000","last_clean":"2024-09-19T21:34:38.161294+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:34:38.161294+0000","last_undegraded":"2024-09-19T21:34:38.161294+0000","last_fullsized":"2024-09-19T21:34:38.161294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":6144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039786+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:37.039786+0000","last_peered":"2024-09-19T21:34:37.039786+0000","last_clean":"2024-09-19T21:34:37.039786+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:37.039786+0000","last_undegraded":"2024-09-19T21:34:37.039786+0000","last_fullsized":"2024-09-19T21:34:37.039786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"30'11","reported_seq":46,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.161226+0000","last_change":"2024-09-19T21:34:32.024327+0000","last_active":"2024-09-19T21:34:38.161226+0000","last_peered":"2024-09-19T21:34:38.161226+0000","last_clean":"2024-09-19T21:34:38.161226+0000","last_became_active":"2024-09-19T21:34:30.007041+0000","last_became_peered":"2024-09-19T21:34:30.007041+0000","last_unstale":"2024-09-19T21:34:38.161226+0000","last_undegraded":"2024-09-19T21:34:38.161226+0000","last_fullsized":"2024-09-19T21:34:38.161226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:31:26.974439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":40960,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":204,"num_write":11,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023253+0000","last_change":"2024-09-19T21:34:31.005184+0000","last_active":"2024-09-19T21:34:33.023253+0000","last_peered":"2024-09-19T21:34:33.023253+0000","last_clean":"2024-09-19T21:34:33.023253+0000","last_became_active":"2024-09-19T21:34:31.004633+0000","last_became_peered":"2024-09-19T21:34:31.004633+0000","last_unstale":"2024-09-19T21:34:33.023253+0000","last_undegraded":"2024-09-19T21:34:33.023253+0000","last_fullsized":"2024-09-19T21:34:33.023253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:21.400292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"30'4","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055278+0000","last_change":"2024-09-19T21:34:32.024762+0000","last_active":"2024-09-19T21:34:38.055278+0000","last_peered":"2024-09-19T21:34:38.055278+0000","last_clean":"2024-09-19T21:34:38.055278+0000","last_became_active":"2024-09-19T21:34:30.006903+0000","last_became_peered":"2024-09-19T21:34:30.006903+0000","last_unstale":"2024-09-19T21:34:38.055278+0000","last_undegraded":"2024-09-19T21:34:38.055278+0000","last_fullsized":"2024-09-19T21:34:38.055278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.226419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"30'54","reported_seq":75,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.154724+0000","last_change":"2024-09-19T21:34:32.006802+0000","last_active":"2024-09-19T21:34:38.154724+0000","last_peered":"2024-09-19T21:34:38.154724+0000","last_clean":"2024-09-19T21:34:38.154724+0000","last_became_active":"2024-09-19T21:34:30.011547+0000","last_became_peered":"2024-09-19T21:34:30.011547+0000","last_unstale":"2024-09-19T21:34:38.154724+0000","last_undegraded":"2024-09-19T21:34:38.154724+0000","last_fullsized":"2024-09-19T21:34:38.154724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:45:47.281981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":182,"num_write":83,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.045103+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:37.045103+0000","last_peered":"2024-09-19T21:34:37.045103+0000","last_clean":"2024-09-19T21:34:37.045103+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:37.045103+0000","last_undegraded":"2024-09-19T21:34:37.045103+0000","last_fullsized":"2024-09-19T21:34:37.045103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'10","reported_seq":33,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.154539+0000","last_change":"2024-09-19T21:34:32.011746+0000","last_active":"2024-09-19T21:34:38.154539+0000","last_peered":"2024-09-19T21:34:38.154539+0000","last_clean":"2024-09-19T21:34:38.154539+0000","last_became_active":"2024-09-19T21:34:30.013347+0000","last_became_peered":"2024-09-19T21:34:30.013347+0000","last_unstale":"2024-09-19T21:34:38.154539+0000","last_undegraded":"2024-09-19T21:34:38.154539+0000","last_fullsized":"2024-09-19T21:34:38.154539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.129809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":16,"num_write":10,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023328+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:33.023328+0000","last_peered":"2024-09-19T21:34:33.023328+0000","last_clean":"2024-09-19T21:34:33.023328+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:33.023328+0000","last_undegraded":"2024-09-19T21:34:33.023328+0000","last_fullsized":"2024-09-19T21:34:33.023328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"28'53","reported_seq":632,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.048646+0000","last_change":"2024-09-19T21:34:32.024303+0000","last_active":"2024-09-19T21:34:37.048646+0000","last_peered":"2024-09-19T21:34:37.048646+0000","last_clean":"2024-09-19T21:34:37.048646+0000","last_became_active":"2024-09-19T21:34:30.009730+0000","last_became_peered":"2024-09-19T21:34:30.009730+0000","last_unstale":"2024-09-19T21:34:37.048646+0000","last_undegraded":"2024-09-19T21:34:37.048646+0000","last_fullsized":"2024-09-19T21:34:37.048646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1167,"num_read_kb":1002,"num_write":28,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.041987+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:37.041987+0000","last_peered":"2024-09-19T21:34:37.041987+0000","last_clean":"2024-09-19T21:34:37.041987+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:37.041987+0000","last_undegraded":"2024-09-19T21:34:37.041987+0000","last_fullsized":"2024-09-19T21:34:37.041987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"28'8","reported_seq":43,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-19T21:34:36.714367+0000","last_change":"2024-09-19T21:34:32.005130+0000","last_active":"2024-09-19T21:34:36.714367+0000","last_peered":"2024-09-19T21:34:36.714367+0000","last_clean":"2024-09-19T21:34:36.714367+0000","last_became_active":"2024-09-19T21:34:30.009669+0000","last_became_peered":"2024-09-19T21:34:30.009669+0000","last_unstale":"2024-09-19T21:34:36.714367+0000","last_undegraded":"2024-09-19T21:34:36.714367+0000","last_fullsized":"2024-09-19T21:34:36.714367+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:20:42.350074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":69,"num_write":8,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039552+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:37.039552+0000","last_peered":"2024-09-19T21:34:37.039552+0000","last_clean":"2024-09-19T21:34:37.039552+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:37.039552+0000","last_undegraded":"2024-09-19T21:34:37.039552+0000","last_fullsized":"2024-09-19T21:34:37.039552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.039634+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:37.039634+0000","last_peered":"2024-09-19T21:34:37.039634+0000","last_clean":"2024-09-19T21:34:37.039634+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:37.039634+0000","last_undegraded":"2024-09-19T21:34:37.039634+0000","last_fullsized":"2024-09-19T21:34:37.039634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"30'5","reported_seq":1073,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.153697+0000","last_change":"2024-09-19T21:34:32.024996+0000","last_active":"2024-09-19T21:34:38.153697+0000","last_peered":"2024-09-19T21:34:38.153697+0000","last_clean":"2024-09-19T21:34:38.153697+0000","last_became_active":"2024-09-19T21:34:30.009636+0000","last_became_peered":"2024-09-19T21:34:30.009636+0000","last_unstale":"2024-09-19T21:34:38.153697+0000","last_undegraded":"2024-09-19T21:34:38.153697+0000","last_fullsized":"2024-09-19T21:34:38.153697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:11.666768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":2233344,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1050,"num_read_kb":2281,"num_write":5,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"29'6","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.947870+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:34:37.947870+0000","last_peered":"2024-09-19T21:34:37.947870+0000","last_clean":"2024-09-19T21:34:37.947870+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:34:37.947870+0000","last_undegraded":"2024-09-19T21:34:37.947870+0000","last_fullsized":"2024-09-19T21:34:37.947870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"30'17","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.160187+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:34:38.160187+0000","last_peered":"2024-09-19T21:34:38.160187+0000","last_clean":"2024-09-19T21:34:38.160187+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:34:38.160187+0000","last_undegraded":"2024-09-19T21:34:38.160187+0000","last_fullsized":"2024-09-19T21:34:38.160187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":20,"num_write":17,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-19T21:34:33.023176+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:34:33.023176+0000","last_peered":"2024-09-19T21:34:33.023176+0000","last_clean":"2024-09-19T21:34:33.023176+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:34:33.023176+0000","last_undegraded":"2024-09-19T21:34:33.023176+0000","last_fullsized":"2024-09-19T21:34:33.023176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"27'4","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.045295+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:34:37.045295+0000","last_peered":"2024-09-19T21:34:37.045295+0000","last_clean":"2024-09-19T21:34:37.045295+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:34:37.045295+0000","last_undegraded":"2024-09-19T21:34:37.045295+0000","last_fullsized":"2024-09-19T21:34:37.045295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":1261056,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":45,"num_write":4,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"29'2","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.654360+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:34:37.654360+0000","last_peered":"2024-09-19T21:34:37.654360+0000","last_clean":"2024-09-19T21:34:37.654360+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:34:37.654360+0000","last_undegraded":"2024-09-19T21:34:37.654360+0000","last_fullsized":"2024-09-19T21:34:37.654360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"30'16","reported_seq":59,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.150967+0000","last_change":"2024-09-19T21:34:32.024576+0000","last_active":"2024-09-19T21:34:38.150967+0000","last_peered":"2024-09-19T21:34:38.150967+0000","last_clean":"2024-09-19T21:34:38.150967+0000","last_became_active":"2024-09-19T21:34:30.006185+0000","last_became_peered":"2024-09-19T21:34:30.006185+0000","last_unstale":"2024-09-19T21:34:38.150967+0000","last_undegraded":"2024-09-19T21:34:38.150967+0000","last_fullsized":"2024-09-19T21:34:38.150967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:08:25.609727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":223,"num_write":16,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"30'161","reported_seq":182,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.222148+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:34:38.222148+0000","last_peered":"2024-09-19T21:34:38.222148+0000","last_clean":"2024-09-19T21:34:38.222148+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:34:38.222148+0000","last_undegraded":"2024-09-19T21:34:38.222148+0000","last_fullsized":"2024-09-19T21:34:38.222148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":42557,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":848,"num_read_kb":692,"num_write":286,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"30'4","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.133090+0000","last_change":"2024-09-19T21:34:32.000772+0000","last_active":"2024-09-19T21:34:38.133090+0000","last_peered":"2024-09-19T21:34:38.133090+0000","last_clean":"2024-09-19T21:34:38.133090+0000","last_became_active":"2024-09-19T21:34:30.007945+0000","last_became_peered":"2024-09-19T21:34:30.007945+0000","last_unstale":"2024-09-19T21:34:38.133090+0000","last_undegraded":"2024-09-19T21:34:38.133090+0000","last_fullsized":"2024-09-19T21:34:38.133090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:20:07.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"30'8","reported_seq":55,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.054603+0000","last_change":"2024-09-19T21:34:32.023713+0000","last_active":"2024-09-19T21:34:38.054603+0000","last_peered":"2024-09-19T21:34:38.054603+0000","last_clean":"2024-09-19T21:34:38.054603+0000","last_became_active":"2024-09-19T21:34:30.003969+0000","last_became_peered":"2024-09-19T21:34:30.003969+0000","last_unstale":"2024-09-19T21:34:38.054603+0000","last_undegraded":"2024-09-19T21:34:38.054603+0000","last_fullsized":"2024-09-19T21:34:38.054603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:36.748615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":280,"num_write":8,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T21:34:37.055666+0000","last_change":"2024-09-19T21:34:32.011754+0000","last_active":"2024-09-19T21:34:37.055666+0000","last_peered":"2024-09-19T21:34:37.055666+0000","last_clean":"2024-09-19T21:34:37.055666+0000","last_became_active":"2024-09-19T21:34:30.012126+0000","last_became_peered":"2024-09-19T21:34:30.012126+0000","last_unstale":"2024-09-19T21:34:37.055666+0000","last_undegraded":"2024-09-19T21:34:37.055666+0000","last_fullsized":"2024-09-19T21:34:37.055666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:32.355336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"30'9","reported_seq":153,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-19T21:34:38.055683+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:34:38.055683+0000","last_peered":"2024-09-19T21:34:38.055683+0000","last_clean":"2024-09-19T21:34:38.055683+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:34:38.055683+0000","last_undegraded":"2024-09-19T21:34:38.055683+0000","last_fullsized":"2024-09-19T21:34:38.055683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":3537408,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":674,"num_write":9,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":8028364,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5854,"num_read_kb":9173,"num_write":649,"num_write_kb":2625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":267160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":555,"ondisk_log_size":555,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":946176,"data_stored":733999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":417792,"data_stored":205613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":245034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":974848,"data_stored":760509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":598016,"data_stored":362883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":3,"up_from":12,"seq":51539607560,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":307200,"data_stored":115137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":74347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":30270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":23102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":49198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":16419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:41.949 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-19T21:34:41.950 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:41.936+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:41.950 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:34:41.950 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-19T21:34:41.950 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:34:41.950 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:34:41.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-19T21:34:42.045 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:42.041+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:42.085 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:42.085 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T21:34:42.085 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:42.093 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:34:42.113 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:42.113 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:42.113 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:42.114 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:42.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:42.146 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:42.140+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:42.243 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:42.243 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:42.246 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:42.240+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:42.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:42.347 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:42.340+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:42.427 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:42.427 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1643191900000001, 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.428 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:34:42.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.162992721, 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:34:42.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1629612469999999, 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:42.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:34:42.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.441 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1629222930000001, 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:42.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:42.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1628812500000001, 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:34:42.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.450 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4822 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153637706, 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021034206999999999, 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4822 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268249+0000", 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.316e-06 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268252+0000", 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268255+0000", 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9629999999999998e-06 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289149+0000", 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.020894302 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289216+0000", 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7269999999999998e-05 2024-09-19T21:34:42.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289282+0000", 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5901999999999993e-05 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:42.455 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153628866, 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:34:42.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:42.460 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1536186370000001, 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:34:42.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:34:42.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153609195, 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:42.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:34:42.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153590755, 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:34:42.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153580147, 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:42.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1535714079999999, 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:42.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1535591590000001, 2024-09-19T21:34:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:34:42.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.487 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1533603450000001, 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:42.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.492 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153342482, 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:42.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:34:42.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1533259359999999, 2024-09-19T21:34:42.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:34:42.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:34:42.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153309591, 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:34:42.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:34:42.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153292714, 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:34:42.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.510 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.153274527, 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:42.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:34:42.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1532559980000001, 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:42.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:42.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:34:42.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:34:42.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:42.519 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:42.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:42.521 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:42.512+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:42.622 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:42.617+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:42.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:42.682 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:42.682 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:42.682 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:42.692 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:42.704 INFO:teuthology.orchestra.run.smithi122.stdout:502 write 0xaff9400 thru 0xaff9dfb (0x9fc bytes) 2024-09-19T21:34:42.705 INFO:teuthology.orchestra.run.smithi122.stdout:504 read 0x80f4200 thru 0x80f7b54 (0x3955 bytes) 2024-09-19T21:34:42.706 INFO:teuthology.orchestra.run.smithi122.stdout:505 read 0x98f2200 thru 0x98fa1e2 (0x7fe3 bytes) 2024-09-19T21:34:42.722 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:42.717+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:42.766 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:42.771 INFO:teuthology.orchestra.run.smithi122.stdout:507 trunc from 0xdf6a000 to 0x2646400 2024-09-19T21:34:42.791 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:42.801 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:42.801 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:42.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:42.817 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:42.823 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:42.816+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:42.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:42.877 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:42.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:42.912 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:42.925 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:42.916+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:42.947 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:42.947 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:42.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:43.025 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:43.016+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:43.080 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.081 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:43.105 INFO:teuthology.orchestra.run.smithi122.stdout:509 read 0x128aa00 thru 0x129974a (0xed4b bytes) 2024-09-19T21:34:43.108 INFO:teuthology.orchestra.run.smithi122.stdout:511 read 0x19ea600 thru 0x19f8b6e (0xe56f bytes) 2024-09-19T21:34:43.111 INFO:teuthology.orchestra.run.smithi122.stdout:513 trunc from 0x2646400 to 0x3111c00 2024-09-19T21:34:43.125 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:43.121+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:43.176 INFO:teuthology.orchestra.run.smithi122.stdout:515 write 0xdf79800 thru 0xdf81caa (0x84ab bytes) 2024-09-19T21:34:43.178 INFO:teuthology.orchestra.run.smithi122.stdout:517 write 0x8e85600 thru 0x8e8a81b (0x521c bytes) 2024-09-19T21:34:43.179 INFO:teuthology.orchestra.run.smithi122.stdout:518 write 0x9a31c00 thru 0x9a392ef (0x76f0 bytes) 2024-09-19T21:34:43.180 INFO:teuthology.orchestra.run.smithi122.stdout:519 write 0xc1cb200 thru 0xc1d34b4 (0x82b5 bytes) 2024-09-19T21:34:43.211 INFO:teuthology.orchestra.run.smithi122.stdout:520 trunc from 0xdf81e00 to 0x124ee00 2024-09-19T21:34:43.227 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:43.220+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:43.257 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.258 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:43.315 INFO:teuthology.orchestra.run.smithi122.stdout:522 write 0xe9ca200 thru 0xe9d25fa (0x83fb bytes) 2024-09-19T21:34:43.317 INFO:teuthology.orchestra.run.smithi122.stdout:523 write 0xe33b400 thru 0xe3473ba (0xbfbb bytes) 2024-09-19T21:34:43.318 INFO:teuthology.orchestra.run.smithi122.stdout:524 write 0x5c38600 thru 0x5c3b60c (0x300d bytes) 2024-09-19T21:34:43.319 INFO:teuthology.orchestra.run.smithi122.stdout:525 write 0xb985600 thru 0xb992b0c (0xd50d bytes) 2024-09-19T21:34:43.320 INFO:teuthology.orchestra.run.smithi122.stdout:527 write 0x792c600 thru 0x793b6ef (0xf0f0 bytes) 2024-09-19T21:34:43.321 INFO:teuthology.orchestra.run.smithi122.stdout:528 read 0x15d4e00 thru 0x15d78f8 (0x2af9 bytes) 2024-09-19T21:34:43.327 INFO:teuthology.orchestra.run.smithi122.stdout:531 trunc from 0xe9d2600 to 0xc000800 2024-09-19T21:34:43.328 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:43.325+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:43.363 INFO:teuthology.orchestra.run.smithi122.stdout:532 read 0x3b6200 thru 0x3b70a5 (0xea6 bytes) 2024-09-19T21:34:43.367 INFO:teuthology.orchestra.run.smithi122.stdout:535 read 0x506f800 thru 0x50724c7 (0x2cc8 bytes) 2024-09-19T21:34:43.368 INFO:teuthology.orchestra.run.smithi122.stdout:536 write 0x3814200 thru 0x3817883 (0x3684 bytes) 2024-09-19T21:34:43.369 INFO:teuthology.orchestra.run.smithi122.stdout:539 read 0x7b3a800 thru 0x7b3f247 (0x4a48 bytes) 2024-09-19T21:34:43.371 INFO:teuthology.orchestra.run.smithi122.stdout:540 trunc from 0xc000800 to 0xb97a400 2024-09-19T21:34:43.380 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.380 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:43.397 INFO:teuthology.orchestra.run.smithi122.stdout:544 read 0xa1ada00 thru 0xa1b8694 (0xac95 bytes) 2024-09-19T21:34:43.401 INFO:teuthology.orchestra.run.smithi122.stdout:545 write 0x4c9c000 thru 0x4c9cb6d (0xb6e bytes) 2024-09-19T21:34:43.402 INFO:teuthology.orchestra.run.smithi122.stdout:549 read 0x1e00400 thru 0x1e0836c (0x7f6d bytes) 2024-09-19T21:34:43.404 INFO:teuthology.orchestra.run.smithi122.stdout:552 write 0x7edb400 thru 0x7ee7331 (0xbf32 bytes) 2024-09-19T21:34:43.416 INFO:teuthology.orchestra.run.smithi122.stdout:553 trunc from 0xb97a400 to 0x70e5400 2024-09-19T21:34:43.429 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:43.420+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:43.448 INFO:teuthology.orchestra.run.smithi122.stdout:555 read 0x489b400 thru 0x489ff6d (0x4b6e bytes) 2024-09-19T21:34:43.451 INFO:teuthology.orchestra.run.smithi122.stdout:557 read 0x19ea600 thru 0x19f2986 (0x8387 bytes) 2024-09-19T21:34:43.468 INFO:teuthology.orchestra.run.smithi122.stdout:558 trunc from 0x70e5400 to 0xe3b7000 2024-09-19T21:34:43.506 INFO:teuthology.orchestra.run.smithi122.stdout:560 trunc from 0xe3b7000 to 0x6ed5600 2024-09-19T21:34:43.518 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.518 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:43.531 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:43.524+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:43.538 INFO:teuthology.orchestra.run.smithi122.stdout:561 write 0x521fa00 thru 0x522246a (0x2a6b bytes) 2024-09-19T21:34:43.542 INFO:teuthology.orchestra.run.smithi122.stdout:562 punch from 0x2ee1600 to 0x2ee2400, (0xe00 bytes) 2024-09-19T21:34:43.543 INFO:teuthology.orchestra.run.smithi122.stdout:563 read 0x456a00 thru 0x460d5d (0xa35e bytes) 2024-09-19T21:34:43.558 INFO:teuthology.orchestra.run.smithi122.stdout:564 trunc from 0x6ed5600 to 0x169de00 2024-09-19T21:34:43.587 INFO:teuthology.orchestra.run.smithi122.stdout:565 read 0x1661200 thru 0x1670216 (0xf017 bytes) 2024-09-19T21:34:43.628 INFO:teuthology.orchestra.run.smithi122.stdout:567 write 0xad4e000 thru 0xad5c82a (0xe82b bytes) 2024-09-19T21:34:43.631 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:43.624+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:43.632 INFO:teuthology.orchestra.run.smithi122.stdout:568 trunc from 0xad5ca00 to 0x9fd9e00 2024-09-19T21:34:43.640 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.640 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:43.657 INFO:teuthology.orchestra.run.smithi122.stdout:570 punch from 0x1fb5800 to 0x1fbea00, (0x9200 bytes) 2024-09-19T21:34:43.660 INFO:teuthology.orchestra.run.smithi122.stdout:572 write 0x2780e00 thru 0x278de7e (0xd07f bytes) 2024-09-19T21:34:43.661 INFO:teuthology.orchestra.run.smithi122.stdout:574 write 0x38e9600 thru 0x38ead96 (0x1797 bytes) 2024-09-19T21:34:43.662 INFO:teuthology.orchestra.run.smithi122.stdout:575 write 0x28eec00 thru 0x28fcb08 (0xdf09 bytes) 2024-09-19T21:34:43.663 INFO:teuthology.orchestra.run.smithi122.stdout:577 read 0x8868a00 thru 0x886d458 (0x4a59 bytes) 2024-09-19T21:34:43.664 INFO:teuthology.orchestra.run.smithi122.stdout:578 read 0x957f800 thru 0x95800d9 (0x8da bytes) 2024-09-19T21:34:43.665 INFO:teuthology.orchestra.run.smithi122.stdout:583 read 0x65dbc00 thru 0x65e9740 (0xdb41 bytes) 2024-09-19T21:34:43.666 INFO:teuthology.orchestra.run.smithi122.stdout:584 read 0x205fe00 thru 0x2062262 (0x2463 bytes) 2024-09-19T21:34:43.666 INFO:teuthology.orchestra.run.smithi122.stdout:585 read 0x4888e00 thru 0x48898bd (0xabe bytes) 2024-09-19T21:34:43.668 INFO:teuthology.orchestra.run.smithi122.stdout:586 read 0x87f4e00 thru 0x87fdc98 (0x8e99 bytes) 2024-09-19T21:34:43.668 INFO:teuthology.orchestra.run.smithi122.stdout:588 read 0x96c800 thru 0x979193 (0xc994 bytes) 2024-09-19T21:34:43.670 INFO:teuthology.orchestra.run.smithi122.stdout:589 read 0x9423800 thru 0x9429153 (0x5954 bytes) 2024-09-19T21:34:43.706 INFO:teuthology.orchestra.run.smithi122.stdout:592 write 0xc6d4800 thru 0xc6d9082 (0x4883 bytes) 2024-09-19T21:34:43.708 INFO:teuthology.orchestra.run.smithi122.stdout:593 punch from 0x2b00800 to 0x2b05c00, (0x5400 bytes) 2024-09-19T21:34:43.709 INFO:teuthology.orchestra.run.smithi122.stdout:595 punch from 0xb0a8800 to 0xb0ad000, (0x4800 bytes) 2024-09-19T21:34:43.709 INFO:teuthology.orchestra.run.smithi122.stdout:597 read 0x1283600 thru 0x128d5d2 (0x9fd3 bytes) 2024-09-19T21:34:43.711 INFO:teuthology.orchestra.run.smithi122.stdout:598 read 0x388b600 thru 0x38987ba (0xd1bb bytes) 2024-09-19T21:34:43.715 INFO:teuthology.orchestra.run.smithi122.stdout:600 write 0x5266600 thru 0x526b572 (0x4f73 bytes) 2024-09-19T21:34:43.716 INFO:teuthology.orchestra.run.smithi122.stdout:601 write 0x37c1200 thru 0x37c587a (0x467b bytes) 2024-09-19T21:34:43.727 INFO:teuthology.orchestra.run.smithi122.stdout:602 trunc from 0xc6d9200 to 0x7f62e00 2024-09-19T21:34:43.732 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:43.725+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:43.757 INFO:teuthology.orchestra.run.smithi122.stdout:603 read 0x5879600 thru 0x5888086 (0xea87 bytes) 2024-09-19T21:34:43.761 INFO:teuthology.orchestra.run.smithi122.stdout:604 read 0x3714000 thru 0x372201d (0xe01e bytes) 2024-09-19T21:34:43.762 INFO:teuthology.orchestra.run.smithi122.stdout:605 read 0x3da3200 thru 0x3db1c79 (0xea7a bytes) 2024-09-19T21:34:43.763 INFO:teuthology.orchestra.run.smithi122.stdout:606 read 0x6b81800 thru 0x6b881f2 (0x69f3 bytes) 2024-09-19T21:34:43.764 INFO:teuthology.orchestra.run.smithi122.stdout:611 punch from 0x1ef6200 to 0x1ef6e00, (0xc00 bytes) 2024-09-19T21:34:43.764 INFO:teuthology.orchestra.run.smithi122.stdout:614 read 0x12dd400 thru 0x12e5b69 (0x876a bytes) 2024-09-19T21:34:43.767 INFO:teuthology.orchestra.run.smithi122.stdout:615 write 0x504b000 thru 0x504db6a (0x2b6b bytes) 2024-09-19T21:34:43.767 INFO:teuthology.orchestra.run.smithi122.stdout:617 punch from 0x634c400 to 0x6352800, (0x6400 bytes) 2024-09-19T21:34:43.768 INFO:teuthology.orchestra.run.smithi122.stdout:620 write 0x6a9a800 thru 0x6aa2ff2 (0x87f3 bytes) 2024-09-19T21:34:43.769 INFO:teuthology.orchestra.run.smithi122.stdout:621 read 0x7423800 thru 0x7428aa0 (0x52a1 bytes) 2024-09-19T21:34:43.771 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.771 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:43.807 INFO:teuthology.orchestra.run.smithi122.stdout:627 write 0xcd4a000 thru 0xcd52f0d (0x8f0e bytes) 2024-09-19T21:34:43.809 INFO:teuthology.orchestra.run.smithi122.stdout:630 read 0x3cc5200 thru 0x3cccc12 (0x7a13 bytes) 2024-09-19T21:34:43.812 INFO:teuthology.orchestra.run.smithi122.stdout:632 trunc from 0xcd53000 to 0xde16600 2024-09-19T21:34:43.832 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:43.829+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:43.835 INFO:teuthology.orchestra.run.smithi122.stdout:634 write 0x846d000 thru 0x847baf2 (0xeaf3 bytes) 2024-09-19T21:34:43.865 INFO:teuthology.orchestra.run.smithi122.stdout:635 trunc from 0xde16600 to 0x1b58200 2024-09-19T21:34:43.902 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:43.902 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:43.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:43.933 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:43.924+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:43.964 INFO:teuthology.orchestra.run.smithi122.stdout:636 write 0xdb60c00 thru 0xdb61095 (0x496 bytes) 2024-09-19T21:34:43.968 INFO:teuthology.orchestra.run.smithi122.stdout:637 write 0x99d9000 thru 0x99e3eb8 (0xaeb9 bytes) 2024-09-19T21:34:43.969 INFO:teuthology.orchestra.run.smithi122.stdout:639 write 0x83b3400 thru 0x83c31c8 (0xfdc9 bytes) 2024-09-19T21:34:43.970 INFO:teuthology.orchestra.run.smithi122.stdout:641 read 0xce86a00 thru 0xce95e0f (0xf410 bytes) 2024-09-19T21:34:43.971 INFO:teuthology.orchestra.run.smithi122.stdout:643 write 0x3ad1600 thru 0x3ad96fa (0x80fb bytes) 2024-09-19T21:34:43.972 INFO:teuthology.orchestra.run.smithi122.stdout:645 write 0xd4dea00 thru 0xd4e03a1 (0x19a2 bytes) 2024-09-19T21:34:43.996 INFO:teuthology.orchestra.run.smithi122.stdout:646 trunc from 0xdb61200 to 0x1475200 2024-09-19T21:34:44.034 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:44.029+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:44.057 INFO:teuthology.orchestra.run.smithi122.stdout:647 read 0xfda600 thru 0xfe1f18 (0x7919 bytes) 2024-09-19T21:34:44.061 INFO:teuthology.orchestra.run.smithi122.stdout:649 read 0x1102400 thru 0x1112269 (0xfe6a bytes) 2024-09-19T21:34:44.083 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.083 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.089 INFO:teuthology.orchestra.run.smithi122.stdout:650 write 0x65ea600 thru 0x65ea603 (0x4 bytes) 2024-09-19T21:34:44.092 INFO:teuthology.orchestra.run.smithi122.stdout:651 flatten 2024-09-19T21:34:44.094 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:44.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:34:44.145 INFO:teuthology.orchestra.run.smithi122.stdout:654 write 0xb506800 thru 0xb50ce56 (0x6657 bytes) 2024-09-19T21:34:44.147 INFO:teuthology.orchestra.run.smithi122.stdout:659 punch from 0x36ef800 to 0x36f8000, (0x8800 bytes) 2024-09-19T21:34:44.150 INFO:teuthology.orchestra.run.smithi122.stdout:661 trunc from 0xb50d000 to 0xa524a00 2024-09-19T21:34:44.158 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:44.152+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:44.175 INFO:teuthology.orchestra.run.smithi122.stdout:662 write 0x4e5b800 thru 0x4e617b3 (0x5fb4 bytes) 2024-09-19T21:34:44.180 INFO:teuthology.orchestra.run.smithi122.stdout:663 punch from 0x7f44200 to 0x7f4c200, (0x8000 bytes) 2024-09-19T21:34:44.180 INFO:teuthology.orchestra.run.smithi122.stdout:664 write 0x760ec00 thru 0x761d602 (0xea03 bytes) 2024-09-19T21:34:44.181 INFO:teuthology.orchestra.run.smithi122.stdout:665 read 0x3452c00 thru 0x3456a55 (0x3e56 bytes) 2024-09-19T21:34:44.182 INFO:teuthology.orchestra.run.smithi122.stdout:666 punch from 0x7e35800 to 0x7e43200, (0xda00 bytes) 2024-09-19T21:34:44.183 INFO:teuthology.orchestra.run.smithi122.stdout:669 read 0x9203400 thru 0x92076bf (0x42c0 bytes) 2024-09-19T21:34:44.183 INFO:teuthology.orchestra.run.smithi122.stdout:670 write 0x6dd4000 thru 0x6ddaa00 (0x6a01 bytes) 2024-09-19T21:34:44.184 INFO:teuthology.orchestra.run.smithi122.stdout:671 read 0x7561200 thru 0x756dadf (0xc8e0 bytes) 2024-09-19T21:34:44.185 INFO:teuthology.orchestra.run.smithi122.stdout:672 punch from 0x5529c00 to 0x5534800, (0xac00 bytes) 2024-09-19T21:34:44.186 INFO:teuthology.orchestra.run.smithi122.stdout:673 read 0x3017c00 thru 0x3024ef8 (0xd2f9 bytes) 2024-09-19T21:34:44.189 INFO:teuthology.orchestra.run.smithi122.stdout:674 read 0x49a0a00 thru 0x49a1c89 (0x128a bytes) 2024-09-19T21:34:44.190 INFO:teuthology.orchestra.run.smithi122.stdout:675 punch from 0xbd9200 to 0xbe6c00, (0xda00 bytes) 2024-09-19T21:34:44.190 INFO:teuthology.orchestra.run.smithi122.stdout:677 write 0x7149e00 thru 0x7153557 (0x9758 bytes) 2024-09-19T21:34:44.191 INFO:teuthology.orchestra.run.smithi122.stdout:678 punch from 0x438e00 to 0x444c00, (0xbe00 bytes) 2024-09-19T21:34:44.192 INFO:teuthology.orchestra.run.smithi122.stdout:679 punch from 0x9352e00 to 0x9362600, (0xf800 bytes) 2024-09-19T21:34:44.208 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.208 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.209 INFO:teuthology.orchestra.run.smithi122.stdout:681 write 0xace2400 thru 0xacedb2a (0xb72b bytes) 2024-09-19T21:34:44.212 INFO:teuthology.orchestra.run.smithi122.stdout:684 punch from 0x153ce00 to 0x154bc00, (0xee00 bytes) 2024-09-19T21:34:44.217 INFO:teuthology.orchestra.run.smithi122.stdout:685 trunc from 0xacedc00 to 0xc3ffe00 2024-09-19T21:34:44.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:44.237 INFO:teuthology.orchestra.run.smithi122.stdout:686 write 0x8113c00 thru 0x81226da (0xeadb bytes) 2024-09-19T21:34:44.237 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:44.228+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:44.240 INFO:teuthology.orchestra.run.smithi122.stdout:687 punch from 0xa8bd200 to 0xa8c2c00, (0x5a00 bytes) 2024-09-19T21:34:44.241 INFO:teuthology.orchestra.run.smithi122.stdout:691 read 0x8330e00 thru 0x833c467 (0xb668 bytes) 2024-09-19T21:34:44.242 INFO:teuthology.orchestra.run.smithi122.stdout:692 punch from 0x4f41e00 to 0x4f4b400, (0x9600 bytes) 2024-09-19T21:34:44.242 INFO:teuthology.orchestra.run.smithi122.stdout:693 read 0x8d12c00 thru 0x8d1b7be (0x8bbf bytes) 2024-09-19T21:34:44.244 INFO:teuthology.orchestra.run.smithi122.stdout:695 write 0x6cb9600 thru 0x6cbf686 (0x6087 bytes) 2024-09-19T21:34:44.246 INFO:teuthology.orchestra.run.smithi122.stdout:696 write 0xb77fc00 thru 0xb784b9c (0x4f9d bytes) 2024-09-19T21:34:44.247 INFO:teuthology.orchestra.run.smithi122.stdout:697 read 0x3a54600 thru 0x3a586dc (0x40dd bytes) 2024-09-19T21:34:44.247 INFO:teuthology.orchestra.run.smithi122.stdout:700 punch from 0xcd1600 to 0xcd1e00, (0x800 bytes) 2024-09-19T21:34:44.250 INFO:teuthology.orchestra.run.smithi122.stdout:701 trunc from 0xc3ffe00 to 0xce79200 2024-09-19T21:34:44.271 INFO:teuthology.orchestra.run.smithi122.stdout:702 trunc from 0xce79200 to 0xaa54600 2024-09-19T21:34:44.291 INFO:teuthology.orchestra.run.smithi122.stdout:704 read 0x947d000 thru 0x9482aa7 (0x5aa8 bytes) 2024-09-19T21:34:44.294 INFO:teuthology.orchestra.run.smithi122.stdout:706 read 0x3274c00 thru 0x32763b9 (0x17ba bytes) 2024-09-19T21:34:44.294 INFO:teuthology.orchestra.run.smithi122.stdout:707 read 0x7173000 thru 0x7175f84 (0x2f85 bytes) 2024-09-19T21:34:44.295 INFO:teuthology.orchestra.run.smithi122.stdout:708 read 0x892c00 thru 0x8963d2 (0x37d3 bytes) 2024-09-19T21:34:44.296 INFO:teuthology.orchestra.run.smithi122.stdout:709 read 0x824aa00 thru 0x82554be (0xaabf bytes) 2024-09-19T21:34:44.306 INFO:teuthology.orchestra.run.smithi122.stdout:712 trunc from 0xaa54600 to 0x6acca00 2024-09-19T21:34:44.335 INFO:teuthology.orchestra.run.smithi122.stdout:713 trunc from 0x6acca00 to 0x7f71600 2024-09-19T21:34:44.336 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:44.329+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:44.338 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.338 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:44.352 INFO:teuthology.orchestra.run.smithi122.stdout:714 read 0x4b6a200 thru 0x4b7806c (0xde6d bytes) 2024-09-19T21:34:44.355 INFO:teuthology.orchestra.run.smithi122.stdout:715 clone 4 order 24 su 65536 sc 13 2024-09-19T21:34:44.437 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:44.432+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:44.450 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.450 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.460 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:44.538 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:44.533+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:44.566 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.566 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.576 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:44.638 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:44.632+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:44.687 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.687 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:44.740 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:44.732+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:44.806 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.806 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:44.840 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:44.837+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:44.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:44.941 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:44.937+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:44.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:45.042 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:45.036+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:45.063 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:45.087 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:45.089 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.089 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:45.089 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:45.089 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:45.103 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:34:45.109 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:45.143 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:45.136+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:45.149 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:45.175 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:45.206 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:45.208 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:45.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:45.212 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:45.213 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.223 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:45.265 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:45.260+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:45.345 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:45.336+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:45.382 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.382 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:45.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.443348133000001, 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:45.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:34:45.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:34:45.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:34:45.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.389 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4788 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1071224040000001, 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053208699999999998, 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4788 2024-09-19T21:34:45.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269024+0000", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269027+0000", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0889999999999998e-06 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269032+0000", 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.31e-06 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269050+0000", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8138000000000002e-05 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322058+0000", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053008161999999998 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322126+0000", 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8269999999999995e-05 2024-09-19T21:34:45.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322233+0000", 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107027 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4789 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 423936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1070918320000001, 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053201143999999999, 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4789 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:45.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269066+0000", 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1392e-05 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269073+0000", 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0500000000000003e-06 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269079+0000", 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322103+0000", 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053024486000000003 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322168+0000", 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949000000000004e-05 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322256+0000", 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8187000000000001e-05 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4790 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1070640039999997, 2024-09-19T21:34:45.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053307251999999999, 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4790 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:45.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269097+0000", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4481000000000001e-05 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0243e-05 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0139999999999998e-06 2024-09-19T21:34:45.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322255+0000", 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053142366000000003 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322323+0000", 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7725999999999995e-05 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322390+0000", 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7800999999999997e-05 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4791 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1070301200000001, 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053280393000000002, 2024-09-19T21:34:45.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4791 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269119+0000", 2024-09-19T21:34:45.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269125+0000", 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5180000000000004e-06 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269129+0000", 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6990000000000002e-06 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322279+0000", 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053149890999999998 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322342+0000", 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2392999999999997e-05 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322397+0000", 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5798999999999999e-05 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4792 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1070141580000001, 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053414182999999997, 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4792 2024-09-19T21:34:45.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269141+0000", 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6810000000000003e-06 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3549999999999998e-06 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322416+0000", 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053270709999999999 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322474+0000", 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8161000000000001e-05 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322547+0000", 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3357999999999998e-05 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4796 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1069985100000004, 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053558802000000003, 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4796 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.414 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269148+0000", 2024-09-19T21:34:45.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0439999999999998e-06 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269155+0000", 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.967e-06 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269167+0000", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1935e-05 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322574+0000", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053407102999999997 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322642+0000", 2024-09-19T21:34:45.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7683999999999998e-05 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322707+0000", 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5421999999999998e-05 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4799 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:45.418 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1069755109999999, 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054302238000000003, 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4799 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269171+0000", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0429999999999999e-06 2024-09-19T21:34:45.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269186+0000", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9840000000000001e-06 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323330+0000", 2024-09-19T21:34:45.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054144172999999997 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323399+0000", 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9543999999999997e-05 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323474+0000", 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4564e-05 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4868 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1069569560000003, 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053685682999999998, 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4868 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269192+0000", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2639999999999998e-06 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269198+0000", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8810000000000001e-06 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:45.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1049999999999997e-06 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322728+0000", 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053525831000000003 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322792+0000", 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4367000000000002e-05 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322876+0000", 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3558000000000002e-05 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4869 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1069409129999999, 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053842947000000002, 2024-09-19T21:34:45.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4869 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269208+0000", 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3879999999999998e-06 2024-09-19T21:34:45.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269214+0000", 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269221+0000", 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.657e-06 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322897+0000", 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053676662 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322966+0000", 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9085999999999996e-05 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323049+0000", 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2742000000000002e-05 2024-09-19T21:34:45.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4870 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.106737141, 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053808917999999997, 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4870 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269225+0000", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998155 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269413+0000", 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018773800000000001 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269420+0000", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7399999999999998e-06 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269424+0000", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9389999999999999e-06 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323071+0000", 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053647087000000003 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323137+0000", 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5810999999999997e-05 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323219+0000", 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2236000000000003e-05 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.436 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4871 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1060961650000003, 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053346843999999997, 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4871 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:45.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269436+0000", 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9993844 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270057+0000", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062110799999999999 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270066+0000", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2709999999999998e-06 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270073+0000", 2024-09-19T21:34:45.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9770000000000003e-06 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323240+0000", 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053167430000000002 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323312+0000", 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1581000000000005e-05 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323398+0000", 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5939999999999994e-05 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4872 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1060676230000004, 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053465749, 2024-09-19T21:34:45.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4872 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270079+0000", 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270082+0000", 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9730000000000002e-06 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270088+0000", 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270093+0000", 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323425+0000", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053332349000000001 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323493+0000", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7828000000000002e-05 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323545+0000", 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2166000000000002e-05 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.445 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4873 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 106496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.10604981, 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053689477999999999, 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4873 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270099+0000", 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270106+0000", 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270111+0000", 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7520000000000004e-06 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323583+0000", 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053471779999999997 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323664+0000", 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1402000000000004e-05 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323787+0000", 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122404 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4874 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 116736~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.450 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1060227349999998, 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053900483999999999, 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4874 2024-09-19T21:34:45.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270131+0000", 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9079999999999998e-06 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270138+0000", 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5699999999999998e-06 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270143+0000", 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323826+0000", 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053683467999999998 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323902+0000", 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5350000000000002e-05 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324025+0000", 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012334600000000001 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4875 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1059869720000002, 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054010483999999997, 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4875 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270177+0000", 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6985000000000001e-05 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270188+0000", 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0993e-05 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270193+0000", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7679999999999997e-06 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324068+0000", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053874248 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324119+0000", 2024-09-19T21:34:45.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1615e-05 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324171+0000", 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440999999999998e-05 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4876 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1057025840000003, 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053834246000000002, 2024-09-19T21:34:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4876 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270198+0000", 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997535 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270450+0000", 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025216000000000003 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270460+0000", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8409999999999995e-06 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270467+0000", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6680000000000004e-06 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324197+0000", 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053730214999999998 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324239+0000", 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1755999999999998e-05 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324279+0000", 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0224e-05 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4877 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1056733059999999, 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054028259000000002, 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4877 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270473+0000", 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270476+0000", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8590000000000001e-06 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270490+0000", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4071e-05 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270496+0000", 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7599999999999999e-06 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324308+0000", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053811953000000003 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324349+0000", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0747999999999997e-05 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324502+0000", 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015331599999999999 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.468 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4878 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1056259580000001, 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054078566000000002, 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4878 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270524+0000", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271662+0000", 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001138114 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271683+0000", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0638000000000001e-05 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324520+0000", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052837566000000002 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324559+0000", 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8782999999999998e-05 2024-09-19T21:34:45.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324600+0000", 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.066e-05 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4881 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1040986759999996, 2024-09-19T21:34:45.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053152144999999998, 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4881 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272051+0000", 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8890000000000002e-06 2024-09-19T21:34:45.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272057+0000", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6030000000000004e-06 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272066+0000", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5540000000000003e-06 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325024+0000", 2024-09-19T21:34:45.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052957551999999998 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325114+0000", 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0708e-05 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325201+0000", 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6236999999999994e-05 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.477 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.478 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.479 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:45.480 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:45.473+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:45.580 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:45.573+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:45.599 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:45.600 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:45.680 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:45.672+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:45.715 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:45.715 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:45.781 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:45.776+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:45.826 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1696 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 452608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.734864+0000", 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0946129979999997, 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013968917000000001, 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.836 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1696 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734864+0000", 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734864+0000", 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734863+0000", 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734867+0000", 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4800000000000001e-06 2024-09-19T21:34:45.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734875+0000", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3960000000000006e-06 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734882+0000", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8519999999999998e-06 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748723+0000", 2024-09-19T21:34:45.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013841397 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748792+0000", 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8985000000000004e-05 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748833+0000", 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0547000000000003e-05 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.840 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1697 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.734888+0000", 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0945890829999998, 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014063284000000001, 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1697 2024-09-19T21:34:45.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734888+0000", 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734888+0000", 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734887+0000", 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734890+0000", 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6960000000000001e-06 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734898+0000", 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7700000000000001e-06 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734904+0000", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6540000000000002e-06 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748849+0000", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013944799000000001 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748907+0000", 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7553000000000003e-05 2024-09-19T21:34:45.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748951+0000", 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4229e-05 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1698 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.734910+0000", 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0945667400000003, 2024-09-19T21:34:45.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014154147000000001, 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1698 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734910+0000", 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734910+0000", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734910+0000", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734913+0000", 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1719999999999999e-06 2024-09-19T21:34:45.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734921+0000", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690000000000004e-06 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.734927+0000", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4030000000000004e-06 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.748970+0000", 2024-09-19T21:34:45.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014042633000000001 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749012+0000", 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2101000000000002e-05 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749064+0000", 2024-09-19T21:34:45.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2419000000000001e-05 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1699 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735065+0000", 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.094412137, 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014189491, 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.850 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1699 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735065+0000", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735065+0000", 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735064+0000", 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735106+0000", 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1761999999999998e-05 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.852 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735120+0000", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4503e-05 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735130+0000", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.842e-06 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749096+0000", 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013965609 2024-09-19T21:34:45.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749184+0000", 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8762000000000002e-05 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749254+0000", 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9893000000000003e-05 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1716 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 286720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735195+0000", 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.094281981, 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014201581, 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1716 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735195+0000", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735195+0000", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735194+0000", 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735200+0000", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2049999999999999e-06 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735215+0000", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4460999999999999e-05 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735224+0000", 2024-09-19T21:34:45.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0909999999999999e-06 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749270+0000", 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014046352 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749326+0000", 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5914e-05 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749396+0000", 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0547000000000001e-05 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1717 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 296960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735508+0000", 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0939684109999996, 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014047927, 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1717 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735508+0000", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735508+0000", 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735507+0000", 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735514+0000", 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0670000000000003e-06 2024-09-19T21:34:45.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735545+0000", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0916e-05 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735585+0000", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9699000000000001e-05 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749422+0000", 2024-09-19T21:34:45.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013836864000000001 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749494+0000", 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1885999999999996e-05 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749556+0000", 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2582000000000005e-05 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.863 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1718 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 323584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735599+0000", 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0938775060000001, 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014061406, 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1718 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735599+0000", 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735599+0000", 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735599+0000", 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735604+0000", 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1039999999999998e-06 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735614+0000", 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8779999999999993e-06 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735621+0000", 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2729999999999997e-06 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749573+0000", 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013952213999999999 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749627+0000", 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3714000000000001e-05 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749661+0000", 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3991e-05 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1719 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 333824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735627+0000", 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0938497849999997, 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014153489, 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1719 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735627+0000", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735627+0000", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735627+0000", 2024-09-19T21:34:45.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735630+0000", 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5200000000000002e-06 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735638+0000", 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8439999999999998e-06 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735656+0000", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8349e-05 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749674+0000", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014017869000000001 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749712+0000", 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7483e-05 2024-09-19T21:34:45.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749781+0000", 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8882999999999998e-05 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1720 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735664+0000", 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0938133399999996, 2024-09-19T21:34:45.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014236530000000001, 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1720 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735664+0000", 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735664+0000", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735663+0000", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735667+0000", 2024-09-19T21:34:45.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.532e-06 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735677+0000", 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0299999999999999e-06 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735683+0000", 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8040000000000004e-06 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749806+0000", 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014122255 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749865+0000", 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9138999999999997e-05 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749900+0000", 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5324000000000003e-05 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.877 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1721 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 370688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735708+0000", 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0937688809999999, 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014328188, 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1721 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735708+0000", 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735708+0000", 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735707+0000", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735712+0000", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2810000000000002e-06 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735721+0000", 2024-09-19T21:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7680000000000003e-06 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735729+0000", 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5870000000000004e-06 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749918+0000", 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014188640000000001 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.749973+0000", 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4914999999999999e-05 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750036+0000", 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3660999999999999e-05 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1722 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735735+0000", 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0937420690000001, 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014490828000000001, 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1722 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735735+0000", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735735+0000", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735734+0000", 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735738+0000", 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5439999999999999e-06 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735745+0000", 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6830000000000008e-06 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735752+0000", 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5869999999999998e-06 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750064+0000", 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014311648 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750163+0000", 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9327999999999995e-05 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750226+0000", 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2762000000000007e-05 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1723 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 407552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735757+0000", 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0937196379999996, 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014592081999999999, 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1723 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735757+0000", 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735757+0000", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735757+0000", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735760+0000", 2024-09-19T21:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8899999999999999e-06 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735768+0000", 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8350000000000004e-06 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735773+0000", 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9390000000000002e-06 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750241+0000", 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014467496 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750313+0000", 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2146999999999999e-05 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750349+0000", 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6185e-05 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.891 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1724 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 434176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735779+0000", 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0936978860000002, 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014681157, 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1724 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.892 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735779+0000", 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735779+0000", 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735779+0000", 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735782+0000", 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.269e-06 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735791+0000", 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8340000000000005e-06 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735798+0000", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1069999999999999e-06 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750364+0000", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014565899 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750419+0000", 2024-09-19T21:34:45.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5769000000000001e-05 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750460+0000", 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0726000000000003e-05 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1725 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 444416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735804+0000", 2024-09-19T21:34:45.896 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0936732129999998, 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014802634, 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1725 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735804+0000", 2024-09-19T21:34:45.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735804+0000", 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735803+0000", 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735807+0000", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4510000000000001e-06 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735815+0000", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3590000000000008e-06 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735822+0000", 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9450000000000004e-06 2024-09-19T21:34:45.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750495+0000", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014672757999999999 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750540+0000", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5698000000000002e-05 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750606+0000", 2024-09-19T21:34:45.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5938999999999993e-05 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1726 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735827+0000", 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0936495109999997, 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014916478, 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1726 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735827+0000", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735827+0000", 2024-09-19T21:34:45.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735827+0000", 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735830+0000", 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0230000000000001e-06 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735838+0000", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7200000000000006e-06 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735844+0000", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3899999999999998e-06 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750640+0000", 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014796254999999999 2024-09-19T21:34:45.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750695+0000", 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4808000000000001e-05 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750744+0000", 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8741e-05 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1727 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735850+0000", 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.093627326, 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015025739, 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1727 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735850+0000", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735850+0000", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735849+0000", 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735869+0000", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9443e-05 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735878+0000", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3840000000000003e-06 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735891+0000", 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3077e-05 2024-09-19T21:34:45.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750759+0000", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014867684000000001 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750808+0000", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9267000000000001e-05 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750875+0000", 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7329999999999994e-05 2024-09-19T21:34:45.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1728 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735897+0000", 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0935793800000004, 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015209488, 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1728 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735897+0000", 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735897+0000", 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735897+0000", 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735901+0000", 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.749e-06 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735910+0000", 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8820000000000007e-06 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735924+0000", 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.433e-05 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.750915+0000", 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014991328999999999 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.751015+0000", 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9732e-05 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.751107+0000", 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2033000000000005e-05 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:1729 3.1d 3:b9f2155a:::rbd_data.1106b3d15a22.0000000000000009:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:40.735930+0000", 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0935465879999997, 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015290556, 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1729 2024-09-19T21:34:45.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735930+0000", 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735930+0000", 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735930+0000", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735940+0000", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0688e-05 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735949+0000", 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0059999999999998e-06 2024-09-19T21:34:45.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.735958+0000", 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0639999999999994e-06 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.751129+0000", 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.01517138 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.751174+0000", 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5037000000000003e-05 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:40.751221+0000", 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6888999999999998e-05 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:45.919 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.6020357709999997, 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:34:45.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.923 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3063 3.16 3:6c1d7bda:::rbd_data.11064af7243d.0000000000000020:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.229508+0000", 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.5999683610000002, 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.014602997, 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:45.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3063 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229508+0000", 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229508+0000", 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229507+0000", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229512+0000", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0429999999999998e-06 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229523+0000", 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0881e-05 2024-09-19T21:34:45.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229530+0000", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8759999999999999e-06 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229575+0000", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4796999999999998e-05 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229656+0000", 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1082999999999996e-05 2024-09-19T21:34:45.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 3", 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.229833+0000", 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017666399999999999 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.231753+0000", 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019201680000000001 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:45.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.244028+0000", 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012275062 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.244074+0000", 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.562e-05 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.244111+0000", 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7852000000000003e-05 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.929 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:45.930 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:45.930 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:45.931 INFO:teuthology.orchestra.run.smithi122.stdout:717 read 0x6589000 thru 0x6595588 (0xc589 bytes) 2024-09-19T21:34:45.931 INFO:teuthology.orchestra.run.smithi122.stdout:719 write 0xe00000 thru 0xe0b90a (0xb90b bytes) 2024-09-19T21:34:45.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:45.932 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:45.925+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:46.019 INFO:teuthology.orchestra.run.smithi122.stdout:722 write 0xeabb800 thru 0xeabdd6a (0x256b bytes) 2024-09-19T21:34:46.020 INFO:teuthology.orchestra.run.smithi122.stdout:723 read 0x4dc800 thru 0x4e2ce8 (0x64e9 bytes) 2024-09-19T21:34:46.021 INFO:teuthology.orchestra.run.smithi122.stdout:725 read 0x5157400 thru 0x515d563 (0x6164 bytes) 2024-09-19T21:34:46.032 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:46.024+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:46.058 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.058 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:46.059 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:46.059 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.065 INFO:teuthology.orchestra.run.smithi122.stdout:728 trunc from 0xeabde00 to 0x3e0d400 2024-09-19T21:34:46.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:46.132 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:46.129+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:46.200 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:46.200 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.204 INFO:teuthology.orchestra.run.smithi122.stdout:734 read 0x3cc0800 thru 0x3cc806a (0x786b bytes) 2024-09-19T21:34:46.206 INFO:teuthology.orchestra.run.smithi122.stdout:737 punch from 0x2db8600 to 0x2dbf000, (0x6a00 bytes) 2024-09-19T21:34:46.207 INFO:teuthology.orchestra.run.smithi122.stdout:738 read 0xb28400 thru 0xb2af76 (0x2b77 bytes) 2024-09-19T21:34:46.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:46.233 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:46.229+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:46.244 INFO:teuthology.orchestra.run.smithi122.stdout:739 write 0x9ce7400 thru 0x9cee1be (0x6dbf bytes) 2024-09-19T21:34:46.245 INFO:teuthology.orchestra.run.smithi122.stdout:740 read 0x9c83600 thru 0x9c8be50 (0x8851 bytes) 2024-09-19T21:34:46.246 INFO:teuthology.orchestra.run.smithi122.stdout:741 write 0x1d20000 thru 0x1d2da34 (0xda35 bytes) 2024-09-19T21:34:46.248 INFO:teuthology.orchestra.run.smithi122.stdout:743 trunc from 0x9cee200 to 0x96b9200 2024-09-19T21:34:46.279 INFO:teuthology.orchestra.run.smithi122.stdout:746 read 0x92bf200 thru 0x92c247b (0x327c bytes) 2024-09-19T21:34:46.292 INFO:teuthology.orchestra.run.smithi122.stdout:748 trunc from 0x96b9200 to 0x5899200 2024-09-19T21:34:46.319 INFO:teuthology.orchestra.run.smithi122.stdout:749 read 0x55bd000 thru 0x55c94b3 (0xc4b4 bytes) 2024-09-19T21:34:46.326 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4296 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0687921679999999, 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022203509999999999, 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4296 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:46.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253833+0000", 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.213e-06 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253840+0000", 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5930000000000002e-06 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253845+0000", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3979999999999998e-06 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275901+0000", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022056172999999998 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275964+0000", 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2723000000000002e-05 2024-09-19T21:34:46.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276034+0000", 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9774e-05 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4297 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0687723079999998, 2024-09-19T21:34:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359332999999999, 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4297 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253852+0000", 2024-09-19T21:34:46.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4779999999999998e-06 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253858+0000", 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7740000000000001e-06 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253860+0000", 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276057+0000", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022196344999999999 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276120+0000", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3521000000000004e-05 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276209+0000", 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9135999999999995e-05 2024-09-19T21:34:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4298 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0687577580000003, 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022554268999999998, 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.336 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4298 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253864+0000", 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253873+0000", 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0289999999999994e-06 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253880+0000", 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4729999999999997e-06 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253888+0000", 2024-09-19T21:34:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:34:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276235+0000", 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022346462000000001 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276323+0000", 2024-09-19T21:34:46.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7990000000000003e-05 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276419+0000", 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6291000000000004e-05 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4299 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 335872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0687201599999998, 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022708860000000001, 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.343 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4299 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253905+0000", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.591e-06 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253921+0000", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5282e-05 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253927+0000", 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.003e-06 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276468+0000", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022541017 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276535+0000", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7119000000000006e-05 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276611+0000", 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6347000000000006e-05 2024-09-19T21:34:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4300 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0686924769999999, 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022876461000000001, 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4300 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253932+0000", 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.857e-06 2024-09-19T21:34:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253935+0000", 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253938+0000", 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.441e-06 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276636+0000", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022698381 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276702+0000", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5258000000000004e-05 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276806+0000", 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010488399999999999 2024-09-19T21:34:46.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4301 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0686824780000004, 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023114603000000001, 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4301 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253941+0000", 2024-09-19T21:34:46.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.44e-06 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253944+0000", 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0960000000000001e-06 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253947+0000", 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4109999999999999e-06 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276839+0000", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022892550000000001 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276947+0000", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107793 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277055+0000", 2024-09-19T21:34:46.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010752299999999999 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4302 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0684485160000001, 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023063824, 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4302 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254173+0000", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254179+0000", 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0789999999999997e-06 2024-09-19T21:34:46.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254190+0000", 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0532e-05 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254196+0000", 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6560000000000003e-06 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277079+0000", 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022882731 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277146+0000", 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6934000000000007e-05 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277238+0000", 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1891999999999994e-05 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4303 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0684199420000002, 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023222402, 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4303 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254205+0000", 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8329999999999998e-06 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254212+0000", 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6549999999999998e-06 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254217+0000", 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:34:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277263+0000", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023045955 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277333+0000", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0257000000000002e-05 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277425+0000", 2024-09-19T21:34:46.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1952000000000003e-05 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4304 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0683768569999996, 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023378263, 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.368 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4304 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254245+0000", 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254248+0000", 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2940000000000001e-06 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254256+0000", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254261+0000", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5310000000000001e-06 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277449+0000", 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023187433 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277538+0000", 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9474000000000004e-05 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277624+0000", 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5560000000000001e-05 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4305 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 446464~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0683520380000004, 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033521316000000002, 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4305 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:46.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254273+0000", 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0850000000000001e-06 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254280+0000", 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0720000000000002e-06 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254285+0000", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3839999999999996e-06 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277648+0000", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0233627 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277727+0000", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9128999999999994e-05 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.287792+0000", 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010064559000000001 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4307 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0681715079999998, 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027454658999999999, 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4307 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254450+0000", 2024-09-19T21:34:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254455+0000", 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4560000000000002e-06 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254464+0000", 2024-09-19T21:34:46.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8570000000000007e-06 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254471+0000", 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281760+0000", 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027288818999999999 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281834+0000", 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4019999999999999e-05 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281906+0000", 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1353999999999995e-05 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4309 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0675926589999998, 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027050399999999999, 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4309 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255029+0000", 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255035+0000", 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6500000000000001e-06 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255046+0000", 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1382999999999999e-05 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255061+0000", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.554e-05 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281929+0000", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02686796 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282019+0000", 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9860999999999999e-05 2024-09-19T21:34:46.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282080+0000", 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0891999999999999e-05 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4311 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0674105220000003, 2024-09-19T21:34:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027091388000000001, 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4311 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255211+0000", 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255216+0000", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3319999999999999e-06 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255228+0000", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1202999999999999e-05 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255235+0000", 2024-09-19T21:34:46.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0350000000000004e-06 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282101+0000", 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026866476 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282202+0000", 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282303+0000", 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100883 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4313 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0673715469999996, 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027229189000000001, 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4313 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255250+0000", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255254+0000", 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0629999999999999e-06 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255262+0000", 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-06 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255269+0000", 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4050000000000003e-06 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282326+0000", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027057184000000001 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282403+0000", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6720999999999999e-05 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282480+0000", 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7001000000000004e-05 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.396 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4744 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0535328579999996, 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022372296999999999, 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4744 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269089+0000", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269092+0000", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269102+0000", 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0077e-05 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0289999999999996e-06 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291277+0000", 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022170091999999999 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291356+0000", 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9241999999999998e-05 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291462+0000", 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105609 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4746 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0535111549999998, 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022538171999999999, 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4746 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269123+0000", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0820000000000005e-06 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269128+0000", 2024-09-19T21:34:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0549999999999999e-06 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291488+0000", 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359917 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291558+0000", 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0339000000000005e-05 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291649+0000", 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1558000000000007e-05 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4747 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0534895080000002, 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022815518999999999, 2024-09-19T21:34:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4747 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0185e-05 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7169999999999999e-06 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291712+0000", 2024-09-19T21:34:46.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022561748 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291806+0000", 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3826999999999995e-05 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291948+0000", 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014276600000000001 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4749 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0534682609999999, 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023058227000000001, 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4749 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269156+0000", 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2230000000000001e-06 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269165+0000", 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8470000000000002e-06 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269170+0000", 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8489999999999997e-06 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291977+0000", 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022806918999999998 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1191000000000003e-05 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292212+0000", 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016450899999999999 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.415 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4751 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0534483860000003, 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023246788000000001, 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4751 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6450000000000005e-06 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269191+0000", 2024-09-19T21:34:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9550000000000004e-06 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269196+0000", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8500000000000002e-06 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292239+0000", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023042450999999999 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292360+0000", 2024-09-19T21:34:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121804 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292421+0000", 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0467000000000002e-05 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4753 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.0534205300000004, 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023417911999999999, 2024-09-19T21:34:46.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4753 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269204+0000", 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.481e-06 2024-09-19T21:34:46.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269213+0000", 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000005e-06 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269219+0000", 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1919999999999996e-06 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292445+0000", 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023226009999999998 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292517+0000", 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1898999999999997e-05 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292620+0000", 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103198 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.428 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.428 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.430 INFO:teuthology.orchestra.run.smithi122.stdout:750 write 0x827c800 thru 0x8289e7f (0xd680 bytes) 2024-09-19T21:34:46.430 INFO:teuthology.orchestra.run.smithi122.stdout:753 read 0x5900a00 thru 0x590e1f4 (0xd7f5 bytes) 2024-09-19T21:34:46.430 INFO:teuthology.orchestra.run.smithi122.stdout:755 read 0x617fe00 thru 0x6184005 (0x4206 bytes) 2024-09-19T21:34:46.430 INFO:teuthology.orchestra.run.smithi122.stdout:756 punch from 0x436400 to 0x43f600, (0x9200 bytes) 2024-09-19T21:34:46.430 INFO:teuthology.orchestra.run.smithi122.stdout:758 trunc from 0x828a000 to 0x1163e00 2024-09-19T21:34:46.431 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:46.431 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:46.425+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:46.454 INFO:teuthology.orchestra.run.smithi122.stdout:759 write 0x2f2e600 thru 0x2f35fe5 (0x79e6 bytes) 2024-09-19T21:34:46.495 INFO:teuthology.orchestra.run.smithi122.stdout:760 write 0xa126e00 thru 0xa12c95a (0x5b5b bytes) 2024-09-19T21:34:46.496 INFO:teuthology.orchestra.run.smithi122.stdout:762 write 0x7d2600 thru 0x7d91e6 (0x6be7 bytes) 2024-09-19T21:34:46.497 INFO:teuthology.orchestra.run.smithi122.stdout:764 punch from 0x18d6e00 to 0x18e0400, (0x9600 bytes) 2024-09-19T21:34:46.527 INFO:teuthology.orchestra.run.smithi122.stdout:766 write 0xec85200 thru 0xec8856d (0x336e bytes) 2024-09-19T21:34:46.530 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:46.524+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:46.560 INFO:teuthology.orchestra.run.smithi122.stdout:767 trunc from 0xec88600 to 0x8a7400 2024-09-19T21:34:46.589 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.589 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:46.589 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:46.589 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:46.631 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:46.625+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:46.634 INFO:teuthology.orchestra.run.smithi122.stdout:768 write 0x61c200 thru 0x621349 (0x514a bytes) 2024-09-19T21:34:46.667 INFO:teuthology.orchestra.run.smithi122.stdout:769 write 0x30f1000 thru 0x30f1d3a (0xd3b bytes) 2024-09-19T21:34:46.671 INFO:teuthology.orchestra.run.smithi122.stdout:770 read 0x263f800 thru 0x264bbf5 (0xc3f6 bytes) 2024-09-19T21:34:46.700 INFO:teuthology.orchestra.run.smithi122.stdout:771 write 0x5135000 thru 0x5138b65 (0x3b66 bytes) 2024-09-19T21:34:46.701 INFO:teuthology.orchestra.run.smithi122.stdout:772 read 0x49e6a00 thru 0x49f64a2 (0xfaa3 bytes) 2024-09-19T21:34:46.704 INFO:teuthology.orchestra.run.smithi122.stdout:775 read 0x3fd9600 thru 0x3fe3b61 (0xa562 bytes) 2024-09-19T21:34:46.705 INFO:teuthology.orchestra.run.smithi122.stdout:777 punch from 0x4b05000 to 0x4b08c00, (0x3c00 bytes) 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.708 INFO:teuthology.orchestra.run.smithi122.stdout:778 trunc from 0x5138c00 to 0x5e27c00 2024-09-19T21:34:46.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:46.730 INFO:teuthology.orchestra.run.smithi122.stdout:779 read 0x452d800 thru 0x452d837 (0x38 bytes) 2024-09-19T21:34:46.732 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:46.724+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:46.732 INFO:teuthology.orchestra.run.smithi122.stdout:781 read 0x2065e00 thru 0x2070776 (0xa977 bytes) 2024-09-19T21:34:46.734 INFO:teuthology.orchestra.run.smithi122.stdout:782 read 0x119bc00 thru 0x11a7d4b (0xc14c bytes) 2024-09-19T21:34:46.735 INFO:teuthology.orchestra.run.smithi122.stdout:785 read 0x2f52000 thru 0x2f5bd10 (0x9d11 bytes) 2024-09-19T21:34:46.736 INFO:teuthology.orchestra.run.smithi122.stdout:786 read 0x2dd5600 thru 0x2de00e0 (0xaae1 bytes) 2024-09-19T21:34:46.738 INFO:teuthology.orchestra.run.smithi122.stdout:788 read 0x4cda400 thru 0x4ce1f24 (0x7b25 bytes) 2024-09-19T21:34:46.772 INFO:teuthology.orchestra.run.smithi122.stdout:789 write 0xb102200 thru 0xb10288d (0x68e bytes) 2024-09-19T21:34:46.787 INFO:teuthology.orchestra.run.smithi122.stdout:790 trunc from 0xb102a00 to 0x5522e00 2024-09-19T21:34:46.822 INFO:teuthology.orchestra.run.smithi122.stdout:792 read 0x464b200 thru 0x465233b (0x713c bytes) 2024-09-19T21:34:46.825 INFO:teuthology.orchestra.run.smithi122.stdout:794 write 0x3c17c00 thru 0x3c1f613 (0x7a14 bytes) 2024-09-19T21:34:46.825 INFO:teuthology.orchestra.run.smithi122.stdout:795 read 0x4898e00 thru 0x48a28ae (0x9aaf bytes) 2024-09-19T21:34:46.827 INFO:teuthology.orchestra.run.smithi122.stdout:796 punch from 0x4cd600 to 0x4cfc00, (0x2600 bytes) 2024-09-19T21:34:46.833 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:46.828+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:46.860 INFO:teuthology.orchestra.run.smithi122.stdout:799 write 0xb896000 thru 0xb897dec (0x1ded bytes) 2024-09-19T21:34:46.864 INFO:teuthology.orchestra.run.smithi122.stdout:800 read 0x43fbc00 thru 0x4406c2b (0xb02c bytes) 2024-09-19T21:34:46.865 INFO:teuthology.orchestra.run.smithi122.stdout:802 read 0xa7df200 thru 0xa7e3bc8 (0x49c9 bytes) 2024-09-19T21:34:46.866 INFO:teuthology.orchestra.run.smithi122.stdout:805 write 0x2b49a00 thru 0x2b575bb (0xdbbc bytes) 2024-09-19T21:34:46.867 INFO:teuthology.orchestra.run.smithi122.stdout:806 punch from 0x74b6400 to 0x74bda00, (0x7600 bytes) 2024-09-19T21:34:46.873 INFO:teuthology.orchestra.run.smithi122.stdout:809 trunc from 0xb897e00 to 0xd385a00 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4767664600000003, 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:34:46.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4767461530000006, 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:34:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.890 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4767266909999996, 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:34:46.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:46.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4766923799999994, 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:34:46.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4766668799999998, 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:34:46.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4727136509999994, 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.905 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:34:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4726765270000008, 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:34:46.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4726393079999998, 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:34:46.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4725949190000005, 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:34:46.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:34:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4725772320000008, 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:34:46.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4725522820000005, 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:34:46.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4725129910000003, 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:46.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:34:46.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.472475588, 2024-09-19T21:34:46.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:34:46.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:34:46.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4724577589999992, 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:34:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4688089810000005, 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4687844370000001, 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:34:46.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:34:46.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4687658219999999, 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:34:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:34:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4687164159999995, 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:34:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4686980179999995, 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:34:46.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.4686722490000008, 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:46.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:46.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:34:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:34:46.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:46.973 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:46.974 INFO:teuthology.orchestra.run.smithi122.stdout:810 trunc from 0xd385a00 to 0xc1e9600 2024-09-19T21:34:46.975 INFO:teuthology.orchestra.run.smithi122.stdout:814 clone 5 order 24 su 262144 sc 3 2024-09-19T21:34:46.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:46.976 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:46.968+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:47.075 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:47.069+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:47.092 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:47.092 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:47.092 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:47.099 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:34:47.099 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:34:47.099 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-19T21:34:47.099 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-19T21:34:47.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:47.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:34:47.175 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:47.168+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:47.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:47.239 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone4 from 0xc1e9600 (overlap 0x8a7400) to 0x298800 2024-09-19T21:34:47.257 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:47.257 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:47.258 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:47.258 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:47.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:47.276 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:47.268+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:47.368 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:47.376 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:47.368+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:47.380 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:104 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.903788529, 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077086730000000001, 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474560+0000", 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474577+0000", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7283e-05 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474609+0000", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2316e-05 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474622+0000", 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.305e-05 2024-09-19T21:34:47.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474765+0000", 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014250999999999999 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474825+0000", 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0529000000000003e-05 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482210+0000", 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073848660000000003 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482270+0000", 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9786e-05 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:47.385 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.902595325, 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006550213, 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:47.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475753+0000", 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475771+0000", 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8067000000000002e-05 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475808+0000", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6665000000000002e-05 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475825+0000", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7827000000000001e-05 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475942+0000", 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011702 2024-09-19T21:34:47.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475984+0000", 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1585999999999998e-05 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482279+0000", 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062948850000000001 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482305+0000", 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5786999999999999e-05 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.902402966, 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087792770000000003, 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475945+0000", 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476124+0000", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000178949 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476158+0000", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3200000000000001e-05 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476181+0000", 2024-09-19T21:34:47.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3898999999999999e-05 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476764+0000", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058244000000000004 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476831+0000", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6792999999999996e-05 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484666+0000", 2024-09-19T21:34:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078356450000000005 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484726+0000", 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9701000000000003e-05 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.901598177, 2024-09-19T21:34:47.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080060370000000006, 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:47.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476750+0000", 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476768+0000", 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8566000000000001e-05 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476817+0000", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8523000000000003e-05 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476833+0000", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6079999999999999e-05 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477864+0000", 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001031243 2024-09-19T21:34:47.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477897+0000", 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3163000000000001e-05 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484733+0000", 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068351740000000003 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484758+0000", 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5075e-05 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.901015893, 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007490137, 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477332+0000", 2024-09-19T21:34:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477346+0000", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3219e-05 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477374+0000", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8799999999999999e-05 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477388+0000", 2024-09-19T21:34:47.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3129000000000001e-05 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478476+0000", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010886030000000001 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478507+0000", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1083e-05 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484770+0000", 2024-09-19T21:34:47.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062626540000000003 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484824+0000", 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4151999999999999e-05 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.900367368000001, 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068625409999999998, 2024-09-19T21:34:47.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477981+0000", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478037+0000", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6742e-05 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478068+0000", 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0303000000000001e-05 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478094+0000", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5925999999999999e-05 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479202+0000", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011083219999999999 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479240+0000", 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7592999999999997e-05 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484826+0000", 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055858959999999999 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484845+0000", 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9425000000000001e-05 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.407 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.899374218999998, 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066455869999999997, 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478974+0000", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478988+0000", 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4202999999999999e-05 2024-09-19T21:34:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479013+0000", 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978999999999999e-05 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479036+0000", 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2959000000000002e-05 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480591+0000", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015545730000000001 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480637+0000", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5939000000000002e-05 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485516+0000", 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004879899 2024-09-19T21:34:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485621+0000", 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104715 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:621 3.1f 3:fe16a831:::rbd_data.1106b3d15a22.00000000000001a7:head [set-alloc-hint object_size 524288 write_size 524288,write 240640~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.351061132, 2024-09-19T21:34:47.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067125730000000003, 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 621 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027286+0000", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027302+0000", 2024-09-19T21:34:47.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5809999999999999e-05 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027339+0000", 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7036000000000003e-05 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027355+0000", 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5265999999999999e-05 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027495+0000", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014067999999999999 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027888+0000", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039285500000000001 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.028329+0000", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044053700000000001 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.030559+0000", 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022299149999999998 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033825+0000", 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032660319999999999 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033901+0000", 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6384999999999995e-05 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.034001+0000", 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100264 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2992 3.0 3:01a40bc1:::rbd_data.1106b3d15a22.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.2300174510000002, 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066149140000000004, 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2992 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148331+0000", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148340+0000", 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6619999999999994e-06 2024-09-19T21:34:47.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148357+0000", 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7674000000000001e-05 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148371+0000", 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3603e-05 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150715+0000", 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023444820000000002 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150853+0000", 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000137575 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.151152+0000", 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029933299999999999 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153280+0000", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021272610000000001 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154735+0000", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001455837 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154867+0000", 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181099999999999 2024-09-19T21:34:47.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154947+0000", 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9961000000000002e-05 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2156 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write 137216~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.5336389529999996, 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068932300000000002, 2024-09-19T21:34:47.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2156 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844708+0000", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844727+0000", 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8975e-05 2024-09-19T21:34:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844757+0000", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0071000000000001e-05 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844769+0000", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1848000000000001e-05 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844825+0000", 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5488999999999997e-05 2024-09-19T21:34:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844987+0000", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016212099999999999 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.845120+0000", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013307299999999999 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.846757+0000", 2024-09-19T21:34:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016375389999999999 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851484+0000", 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047265140000000002 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851555+0000", 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0994000000000005e-05 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851604+0000", 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9172999999999998e-05 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3027 3.1f 3:f973b0ba:::rbd_data.11064af7243d.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1463847300000003, 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014783325, 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:47.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3027 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231962+0000", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231977+0000", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4780000000000001e-05 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232006+0000", 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9094e-05 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232025+0000", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8771e-05 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232146+0000", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120834 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232377+0000", 2024-09-19T21:34:47.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023080400000000001 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232837+0000", 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046002400000000001 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.243127+0000", 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290509999999999 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246633+0000", 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035057460000000001 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246695+0000", 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1860999999999996e-05 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246748+0000", 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3065 3.1f 3:fb335e39:::rbd_data.11064af7243d.0000000000000021:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1452978939999996, 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014219874, 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3065 2024-09-19T21:34:47.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233050+0000", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233063+0000", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2850999999999999e-05 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233086+0000", 2024-09-19T21:34:47.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3125999999999998e-05 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233106+0000", 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0273000000000001e-05 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236241+0000", 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031351590000000002 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236353+0000", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011176899999999999 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236627+0000", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027411599999999998 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246167+0000", 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095400050000000007 2024-09-19T21:34:47.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247178+0000", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001010628 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247224+0000", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6786000000000002e-05 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247272+0000", 2024-09-19T21:34:47.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7264e-05 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3075 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1452254760000002, 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015477372, 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:47.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3075 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233123+0000", 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233134+0000", 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.046e-05 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233162+0000", 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8530999999999998e-05 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233177+0000", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4894e-05 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239745+0000", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065679969999999999 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239854+0000", 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108999 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.240035+0000", 2024-09-19T21:34:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018081799999999999 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247400+0000", 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073650909999999998 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248505+0000", 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011052779999999999 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248555+0000", 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0175e-05 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248602+0000", 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6343e-05 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.449 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3073 3.10 e33/23 3:0fcc0951:::rbd_data.11064af7243d.000000000000009a:head v 33'31, mlcod=32'26)", 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1434498670000002, 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078372670000000002, 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234899+0000", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234914+0000", 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4630999999999999e-05 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234926+0000", 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1901e-05 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234934+0000", 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4100000000000008e-06 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235023+0000", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8639000000000003e-05 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235048+0000", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5230000000000001e-05 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242558+0000", 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075100929999999998 2024-09-19T21:34:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242737+0000", 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3072 3.1a e33/23 3:5c4f536b:::rbd_data.11064af7243d.0000000000000099:head v 33'32, mlcod=32'26)", 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1432007899999999, 2024-09-19T21:34:47.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074658509999999999, 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235148+0000", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235157+0000", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2169999999999994e-06 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235167+0000", 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-19T21:34:47.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235176+0000", 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1419999999999993e-06 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235252+0000", 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6489999999999994e-05 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235277+0000", 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4624999999999999e-05 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242578+0000", 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073010280000000002 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242615+0000", 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7039999999999998e-05 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "pg_update_log_missing(3.4 epoch 33/23 rep_tid 343 entries 33'27 (0'0) error 3:21918efa:::rbd_data.11064af7243d.0000000000000015:head by client.4554.0:3042 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 32'24)", 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1399954240000003, 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074741240000000004, 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238360+0000", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238379+0000", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8966000000000001e-05 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238393+0000", 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3621000000000001e-05 2024-09-19T21:34:47.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238587+0000", 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019394200000000001 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.245828+0000", 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072414319999999999 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3069 3.4 e33/23 3:203a8a6a:::rbd_data.11064af7243d.0000000000000023:head v 33'29, mlcod=32'24)", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.1399493119999997, 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077039919999999998, 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238408+0000", 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4250000000000001e-06 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238414+0000", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8249999999999999e-06 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238419+0000", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4290000000000003e-06 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238781+0000", 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036266899999999998 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238806+0000", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4320000000000001e-05 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246061+0000", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072551999999999998 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246104+0000", 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3495000000000003e-05 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:5036 3.12 3:4d11eb75:::rbd_data.11064af7243d.00000000000000f4:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0967140909999999, 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070879050000000002, 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 5036 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281634+0000", 2024-09-19T21:34:47.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281642+0000", 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6199999999999999e-06 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281658+0000", 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6283999999999999e-05 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281666+0000", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9899999999999997e-06 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281710+0000", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4520000000000001e-05 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281780+0000", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9342000000000001e-05 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:47.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281891+0000", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111544 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.286723+0000", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048321079999999999 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288632+0000", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019087959999999999 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288684+0000", 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080999999999998e-05 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288724+0000", 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9347000000000001e-05 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:5049 3.1a e33/23 3:5b4ddbac:::rbd_data.11064af7243d.0000000000000000:head v 33'37, mlcod=33'32)", 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:47.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 6.0859588919999998, 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069007460000000001, 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292390+0000", 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292397+0000", 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.294e-06 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292410+0000", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2921999999999999e-05 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292417+0000", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2190000000000001e-06 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294086+0000", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016694780000000001 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294119+0000", 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2636999999999999e-05 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299247+0000", 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051283570000000001 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299292+0000", 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4517000000000001e-05 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4644.0:2591 3.1f 3:f9bcee2b:::rbd_data.1106ee579bc7.000000000000000a:head [stat,zero 12124160~524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.11892719, 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065658369999999997, 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4644", 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/4011091148", 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2591 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.477 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259420+0000", 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259437+0000", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6506000000000001e-05 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259469+0000", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2053e-05 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259490+0000", 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1656000000000001e-05 2024-09-19T21:34:47.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259647+0000", 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015686000000000001 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259866+0000", 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021919299999999999 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:47.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.265988+0000", 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006121919 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:47.481 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:47.482 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:47.482 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:avg 13.6667 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.62467 -> 2.62467 (expected baseline 3.37474) 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.4 with 10 -> 10 pgs (0.731707 -> 0.731707 * mean) 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:oload 1000 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 1 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:34:47.483 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0004 2024-09-19T21:34:47.484 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0036 2024-09-19T21:34:47.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:47.485 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:34:47.485 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-19T21:34:47.486 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-19T21:34:47.486 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:47.486 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:47.486 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:47.487 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:47.476+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:47.583 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:47.577+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:47.649 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:47.649 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:47.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:47.685 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:47.681+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:47.766 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:47.766 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:47.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:47.785 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:47.776+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:34:47.836 INFO:teuthology.orchestra.run.smithi012.stdout:avg 13.6667 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.62467 -> 2.62467 (expected baseline 3.37474) 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.4 with 10 -> 10 pgs (0.731707 -> 0.731707 * mean) 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 0.05 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0004 2024-09-19T21:34:47.837 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0004 2024-09-19T21:34:47.840 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:34:47.851 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-19T21:34:47.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-19T21:34:47.883 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:47.883 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:47.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:47.893 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:47.884+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:47.900 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-19T21:34:47.987 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:47.981+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:48.055 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.056 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:48.088 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:48.081+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:48.119 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:34:48.183 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.183 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.189 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:48.185+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:48.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:avg 13.6667 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.62467 -> 2.62467 (expected baseline 3.37474) 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.4 with 10 -> 10 pgs (0.731707 -> 0.731707 * mean) 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 0.05 2024-09-19T21:34:48.220 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:34:48.221 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0004 2024-09-19T21:34:48.221 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0004 2024-09-19T21:34:48.224 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:34:48.236 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-19T21:34:48.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-19T21:34:48.290 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:48.285+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:48.298 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.299 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:48.380 INFO:teuthology.orchestra.run.smithi122.stdout:815 trunc from 0xc1e9600 to 0x9204000 2024-09-19T21:34:48.390 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:48.385+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:48.442 INFO:teuthology.orchestra.run.smithi122.stdout:816 read 0x4171e00 thru 0x4172b66 (0xd67 bytes) 2024-09-19T21:34:48.445 INFO:teuthology.orchestra.run.smithi122.stdout:817 write 0x3514600 thru 0x3522630 (0xe031 bytes) 2024-09-19T21:34:48.445 INFO:teuthology.orchestra.run.smithi122.stdout:818 read 0x42a5800 thru 0x42ac4fa (0x6cfb bytes) 2024-09-19T21:34:48.446 INFO:teuthology.orchestra.run.smithi122.stdout:819 read 0x2fe2c00 thru 0x2fe971d (0x6b1e bytes) 2024-09-19T21:34:48.447 INFO:teuthology.orchestra.run.smithi122.stdout:820 read 0x6b62800 thru 0x6b725fa (0xfdfb bytes) 2024-09-19T21:34:48.449 INFO:teuthology.orchestra.run.smithi122.stdout:823 read 0x415d800 thru 0x4168b1c (0xb31d bytes) 2024-09-19T21:34:48.450 INFO:teuthology.orchestra.run.smithi122.stdout:825 write 0x71a3e00 thru 0x71aa540 (0x6741 bytes) 2024-09-19T21:34:48.451 INFO:teuthology.orchestra.run.smithi122.stdout:828 read 0x4691400 thru 0x4698ce4 (0x78e5 bytes) 2024-09-19T21:34:48.452 INFO:teuthology.orchestra.run.smithi122.stdout:834 read 0x2a61600 thru 0x2a68a7f (0x7480 bytes) 2024-09-19T21:34:48.453 INFO:teuthology.orchestra.run.smithi122.stdout:837 read 0x4a16e00 thru 0x4a1b84c (0x4a4d bytes) 2024-09-19T21:34:48.454 INFO:teuthology.orchestra.run.smithi122.stdout:838 read 0x8945e00 thru 0x89471c2 (0x13c3 bytes) 2024-09-19T21:34:48.454 INFO:teuthology.orchestra.run.smithi122.stdout:841 read 0xc0ac00 thru 0xc1ab89 (0xff8a bytes) 2024-09-19T21:34:48.456 INFO:teuthology.orchestra.run.smithi122.stdout:842 read 0x2004000 thru 0x200865e (0x465f bytes) 2024-09-19T21:34:48.491 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:48.485+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:48.516 INFO:teuthology.orchestra.run.smithi122.stdout:846 write 0xeaafe00 thru 0xeab0b6f (0xd70 bytes) 2024-09-19T21:34:48.518 INFO:teuthology.orchestra.run.smithi122.stdout:848 write 0x7988200 thru 0x79883f3 (0x1f4 bytes) 2024-09-19T21:34:48.519 INFO:teuthology.orchestra.run.smithi122.stdout:849 write 0x852600 thru 0x857b1f (0x5520 bytes) 2024-09-19T21:34:48.521 INFO:teuthology.orchestra.run.smithi122.stdout:850 write 0xe804a00 thru 0xe80ebab (0xa1ac bytes) 2024-09-19T21:34:48.521 INFO:teuthology.orchestra.run.smithi122.stdout:851 write 0x35b0800 thru 0x35b697d (0x617e bytes) 2024-09-19T21:34:48.522 INFO:teuthology.orchestra.run.smithi122.stdout:853 read 0x8cb3200 thru 0x8cbe753 (0xb554 bytes) 2024-09-19T21:34:48.529 INFO:teuthology.orchestra.run.smithi122.stdout:855 trunc from 0xeab0c00 to 0xc766000 2024-09-19T21:34:48.555 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.555 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.566 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:48.575 INFO:teuthology.orchestra.run.smithi122.stdout:856 write 0x991ec00 thru 0x991f4a9 (0x8aa bytes) 2024-09-19T21:34:48.577 INFO:teuthology.orchestra.run.smithi122.stdout:859 write 0x28c4800 thru 0x28d307b (0xe87c bytes) 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:avg 13.6667 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.62467 -> 2.62467 (expected baseline 3.37474) 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.4 with 10 -> 10 pgs (0.731707 -> 0.731707 * mean) 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:oload 1000 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 1 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0004 2024-09-19T21:34:48.580 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0036 2024-09-19T21:34:48.585 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:34:48.592 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:48.589+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:48.596 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-19T21:34:48.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-19T21:34:48.608 INFO:teuthology.orchestra.run.smithi122.stdout:860 trunc from 0xc766000 to 0x6c21200 2024-09-19T21:34:48.663 INFO:teuthology.orchestra.run.smithi122.stdout:861 write 0x44ec800 thru 0x44fa255 (0xda56 bytes) 2024-09-19T21:34:48.674 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.674 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.682 INFO:teuthology.orchestra.run.smithi122.stdout:864 trunc from 0x6c21200 to 0xe35c000 2024-09-19T21:34:48.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:48.693 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:48.689+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:48.706 INFO:teuthology.orchestra.run.smithi122.stdout:865 clone 6 order 24 su 65536 sc 11 2024-09-19T21:34:48.794 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:48.789+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:48.801 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.802 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:48.895 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:48.889+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:avg 13.6667 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.62467 -> 2.62467 (expected baseline 3.37474) 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.4 with 10 -> 10 pgs (0.731707 -> 0.731707 * mean) 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-19T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:48.900 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:34:48.900 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 0.05 2024-09-19T21:34:48.900 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:34:48.900 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0004 2024-09-19T21:34:48.900 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0004 2024-09-19T21:34:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:34:48.921 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:48.921 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:48.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:48.996 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:48.989+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:49.043 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.043 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:49.097 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:49.093+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:49.164 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.164 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:49.198 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:49.193+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:49.281 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.281 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.287 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone5 from 0xe35c000 (overlap 0x6c21200) to 0x4bb0c00 2024-09-19T21:34:49.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:49.344 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:49.341+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:49.400 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:49.393+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:49.406 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.406 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.416 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:49.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:49.500 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:49.497+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:49.526 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.526 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:49.602 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:49.597+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:49.644 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:49.667 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:49.690 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:49.696 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:49.696 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:49.702 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:49.697+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:49.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:49.713 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:49.734 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:49.757 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:49.759 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:49.760 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:49.803 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:49.797+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:104 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:49.872 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.393352811, 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077086730000000001, 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474561+0000", 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474560+0000", 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474577+0000", 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7283e-05 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474609+0000", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2316e-05 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474622+0000", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.305e-05 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474765+0000", 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014250999999999999 2024-09-19T21:34:49.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.474825+0000", 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0529000000000003e-05 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482210+0000", 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073848660000000003 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482270+0000", 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9786e-05 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.392159607000004, 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006550213, 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475754+0000", 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475753+0000", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475771+0000", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8067000000000002e-05 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475808+0000", 2024-09-19T21:34:49.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6665000000000002e-05 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475825+0000", 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7827000000000001e-05 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475942+0000", 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011702 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475984+0000", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1585999999999998e-05 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482279+0000", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062948850000000001 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.482305+0000", 2024-09-19T21:34:49.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5786999999999999e-05 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.391967248, 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0087792770000000003, 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475947+0000", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.475945+0000", 2024-09-19T21:34:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476124+0000", 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000178949 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476158+0000", 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3200000000000001e-05 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476181+0000", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3898999999999999e-05 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476764+0000", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058244000000000004 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476831+0000", 2024-09-19T21:34:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6792999999999996e-05 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484666+0000", 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078356450000000005 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484726+0000", 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9701000000000003e-05 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:107 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.391162459, 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0080060370000000006, 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476752+0000", 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476750+0000", 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:49.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476768+0000", 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8566000000000001e-05 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476817+0000", 2024-09-19T21:34:49.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8523000000000003e-05 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.476833+0000", 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6079999999999999e-05 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477864+0000", 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001031243 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477897+0000", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3163000000000001e-05 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484733+0000", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068351740000000003 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484758+0000", 2024-09-19T21:34:49.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5075e-05 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:108 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.390580174999997, 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.007490137, 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477334+0000", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477332+0000", 2024-09-19T21:34:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477346+0000", 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3219e-05 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477374+0000", 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8799999999999999e-05 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477388+0000", 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3129000000000001e-05 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478476+0000", 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0010886030000000001 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478507+0000", 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1083e-05 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484770+0000", 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0062626540000000003 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484824+0000", 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4151999999999999e-05 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:109 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.389931650000001, 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068625409999999998, 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477982+0000", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.477981+0000", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478037+0000", 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6742e-05 2024-09-19T21:34:49.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478068+0000", 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0303000000000001e-05 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478094+0000", 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5925999999999999e-05 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479202+0000", 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011083219999999999 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479240+0000", 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7592999999999997e-05 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484826+0000", 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0055858959999999999 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.484845+0000", 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9425000000000001e-05 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4163.0:110 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.388938500999998, 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066455869999999997, 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478976+0000", 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478974+0000", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.478988+0000", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4202999999999999e-05 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479013+0000", 2024-09-19T21:34:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4978999999999999e-05 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.479036+0000", 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2959000000000002e-05 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480591+0000", 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0015545730000000001 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.480637+0000", 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5939000000000002e-05 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485516+0000", 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.004879899 2024-09-19T21:34:49.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:16.485621+0000", 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104715 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:621 3.1f 3:fe16a831:::rbd_data.1106b3d15a22.00000000000001a7:head [set-alloc-hint object_size 524288 write_size 524288,write 240640~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.840625414, 2024-09-19T21:34:49.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0067125730000000003, 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 621 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:49.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027289+0000", 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027286+0000", 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027302+0000", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5809999999999999e-05 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027339+0000", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7036000000000003e-05 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027355+0000", 2024-09-19T21:34:49.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5265999999999999e-05 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027495+0000", 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014067999999999999 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.027888+0000", 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039285500000000001 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:49.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.028329+0000", 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044053700000000001 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.030559+0000", 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0022299149999999998 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033825+0000", 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0032660319999999999 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.033901+0000", 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6384999999999995e-05 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.034001+0000", 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100264 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:2992 3.0 3:01a40bc1:::rbd_data.1106b3d15a22.0000000000000030:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 11.719581733, 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0066149140000000004, 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2992 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.913 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148332+0000", 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148331+0000", 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148340+0000", 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6619999999999994e-06 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148357+0000", 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7674000000000001e-05 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.148371+0000", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3603e-05 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150715+0000", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0023444820000000002 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.150853+0000", 2024-09-19T21:34:49.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000137575 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.151152+0000", 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00029933299999999999 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.153280+0000", 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0021272610000000001 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154735+0000", 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001455837 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154867+0000", 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181099999999999 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:38.154947+0000", 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9961000000000002e-05 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:2156 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [set-alloc-hint object_size 524288 write_size 524288,write 137216~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0232032350000004, 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0068932300000000002, 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2156 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844711+0000", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844708+0000", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844727+0000", 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8975e-05 2024-09-19T21:34:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844757+0000", 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0071000000000001e-05 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844769+0000", 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1848000000000001e-05 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844825+0000", 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5488999999999997e-05 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.844987+0000", 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016212099999999999 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.845120+0000", 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013307299999999999 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.846757+0000", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016375389999999999 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851484+0000", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0047265140000000002 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851555+0000", 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0994000000000005e-05 2024-09-19T21:34:49.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:40.851604+0000", 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9172999999999998e-05 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3027 3.1f 3:f973b0ba:::rbd_data.11064af7243d.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6359490119999993, 2024-09-19T21:34:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014783325, 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3027 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231962+0000", 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231977+0000", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4780000000000001e-05 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232006+0000", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9094e-05 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232025+0000", 2024-09-19T21:34:49.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8771e-05 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232146+0000", 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120834 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232377+0000", 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023080400000000001 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232837+0000", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046002400000000001 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.243127+0000", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290509999999999 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246633+0000", 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035057460000000001 2024-09-19T21:34:49.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246695+0000", 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1860999999999996e-05 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246748+0000", 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3065 3.1f 3:fb335e39:::rbd_data.11064af7243d.0000000000000021:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6348621760000004, 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014219874, 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3065 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233050+0000", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233063+0000", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2850999999999999e-05 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233086+0000", 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3125999999999998e-05 2024-09-19T21:34:49.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233106+0000", 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0273000000000001e-05 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236241+0000", 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031351590000000002 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236353+0000", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011176899999999999 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236627+0000", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027411599999999998 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246167+0000", 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095400050000000007 2024-09-19T21:34:49.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247178+0000", 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001010628 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247224+0000", 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6786000000000002e-05 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247272+0000", 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7264e-05 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3075 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6347897580000001, 2024-09-19T21:34:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015477372, 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3075 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233123+0000", 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233134+0000", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.046e-05 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233162+0000", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8530999999999998e-05 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233177+0000", 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4894e-05 2024-09-19T21:34:49.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239745+0000", 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065679969999999999 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239854+0000", 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108999 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.240035+0000", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018081799999999999 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247400+0000", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073650909999999998 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248505+0000", 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011052779999999999 2024-09-19T21:34:49.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248555+0000", 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0175e-05 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248602+0000", 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6343e-05 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3073 3.10 e33/23 3:0fcc0951:::rbd_data.11064af7243d.000000000000009a:head v 33'31, mlcod=32'26)", 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6330141489999992, 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0078372670000000002, 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234900+0000", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234899+0000", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234914+0000", 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4630999999999999e-05 2024-09-19T21:34:49.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234926+0000", 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1901e-05 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.234934+0000", 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4100000000000008e-06 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235023+0000", 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8639000000000003e-05 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235048+0000", 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5230000000000001e-05 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242558+0000", 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075100929999999998 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242737+0000", 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00017911700000000001 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3072 3.1a e33/23 3:5c4f536b:::rbd_data.11064af7243d.0000000000000099:head v 33'32, mlcod=32'26)", 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6327650719999998, 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074658509999999999, 2024-09-19T21:34:49.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235149+0000", 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235148+0000", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235157+0000", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2169999999999994e-06 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235167+0000", 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235176+0000", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1419999999999993e-06 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235252+0000", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6489999999999994e-05 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.235277+0000", 2024-09-19T21:34:49.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4624999999999999e-05 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242578+0000", 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073010280000000002 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.242615+0000", 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7039999999999998e-05 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "pg_update_log_missing(3.4 epoch 33/23 rep_tid 343 entries 33'27 (0'0) error 3:21918efa:::rbd_data.11064af7243d.0000000000000015:head by client.4554.0:3042 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 32'24)", 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6295597060000002, 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074741240000000004, 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:49.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238354+0000", 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238360+0000", 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5440000000000004e-06 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238379+0000", 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8966000000000001e-05 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238393+0000", 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3621000000000001e-05 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238587+0000", 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019394200000000001 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.245828+0000", 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072414319999999999 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:3069 3.4 e33/23 3:203a8a6a:::rbd_data.11064af7243d.0000000000000023:head v 33'29, mlcod=32'24)", 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.6295135940000005, 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0077039919999999998, 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238400+0000", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238408+0000", 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4250000000000001e-06 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238414+0000", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8249999999999999e-06 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238419+0000", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4290000000000003e-06 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238781+0000", 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036266899999999998 2024-09-19T21:34:49.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.238806+0000", 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4320000000000001e-05 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246061+0000", 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0072551999999999998 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246104+0000", 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3495000000000003e-05 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:5036 3.12 3:4d11eb75:::rbd_data.11064af7243d.00000000000000f4:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5862783730000007, 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070879050000000002, 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 5036 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281636+0000", 2024-09-19T21:34:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281634+0000", 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281642+0000", 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6199999999999999e-06 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281658+0000", 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6283999999999999e-05 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281666+0000", 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9899999999999997e-06 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281710+0000", 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4520000000000001e-05 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281780+0000", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9342000000000001e-05 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281891+0000", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000111544 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.286723+0000", 2024-09-19T21:34:49.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0048321079999999999 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288632+0000", 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0019087959999999999 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288684+0000", 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2080999999999998e-05 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.288724+0000", 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9347000000000001e-05 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.4554.0:5049 3.1a e33/23 3:5b4ddbac:::rbd_data.11064af7243d.0000000000000000:head v 33'37, mlcod=33'32)", 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 8.5755231740000006, 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0069007460000000001, 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292391+0000", 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292390+0000", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292397+0000", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.294e-06 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292410+0000", 2024-09-19T21:34:49.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2921999999999999e-05 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292417+0000", 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2190000000000001e-06 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294086+0000", 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0016694780000000001 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.294119+0000", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2636999999999999e-05 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299247+0000", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0051283570000000001 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.299292+0000", 2024-09-19T21:34:49.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4517000000000001e-05 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4644.0:2591 3.1f 3:f9bcee2b:::rbd_data.1106ee579bc7.000000000000000a:head [stat,zero 12124160~524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.6084914719999999, 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065658369999999997, 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:49.973 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4644", 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/4011091148", 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2591 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:49.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259423+0000", 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259420+0000", 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259437+0000", 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6506000000000001e-05 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259469+0000", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2053e-05 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259490+0000", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1656000000000001e-05 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259647+0000", 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015686000000000001 2024-09-19T21:34:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.259866+0000", 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021919299999999999 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:46.265988+0000", 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.006121919 2024-09-19T21:34:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:49.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:49.978 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:49.981 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:49.973+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:49.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:50.034 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-19T21:34:50.080 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:50.077+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:50.113 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.114 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:50.114 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:50.114 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.124 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:50.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:34:50.148 INFO:teuthology.orchestra.run.smithi122.stdout:866 write 0x9ea5800 thru 0x9eb0f68 (0xb769 bytes) 2024-09-19T21:34:50.149 INFO:teuthology.orchestra.run.smithi122.stdout:867 read 0x4436800 thru 0x4439337 (0x2b38 bytes) 2024-09-19T21:34:50.151 INFO:teuthology.orchestra.run.smithi122.stdout:869 read 0xc9bce00 thru 0xc9bdfac (0x11ad bytes) 2024-09-19T21:34:50.152 INFO:teuthology.orchestra.run.smithi122.stdout:870 write 0x9c06400 thru 0x9c102b1 (0x9eb2 bytes) 2024-09-19T21:34:50.152 INFO:teuthology.orchestra.run.smithi122.stdout:873 write 0x62c0c00 thru 0x62c8850 (0x7c51 bytes) 2024-09-19T21:34:50.154 INFO:teuthology.orchestra.run.smithi122.stdout:875 write 0x7ce5e00 thru 0x7cefbd0 (0x9dd1 bytes) 2024-09-19T21:34:50.156 INFO:teuthology.orchestra.run.smithi122.stdout:879 write 0xadf2a00 thru 0xadf8d54 (0x6355 bytes) 2024-09-19T21:34:50.157 INFO:teuthology.orchestra.run.smithi122.stdout:880 read 0x9669800 thru 0x967560c (0xbe0d bytes) 2024-09-19T21:34:50.158 INFO:teuthology.orchestra.run.smithi122.stdout:881 write 0xc1e6200 thru 0xc1e62b0 (0xb1 bytes) 2024-09-19T21:34:50.158 INFO:teuthology.orchestra.run.smithi122.stdout:886 read 0xa801800 thru 0xa80ce81 (0xb682 bytes) 2024-09-19T21:34:50.159 INFO:teuthology.orchestra.run.smithi122.stdout:887 read 0x4f17a00 thru 0x4f1ab6e (0x316f bytes) 2024-09-19T21:34:50.161 INFO:teuthology.orchestra.run.smithi122.stdout:888 punch from 0x5ae8e00 to 0x5aea200, (0x1400 bytes) 2024-09-19T21:34:50.185 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:50.177+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:50.247 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.247 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:50.247 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:50.247 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:50.247 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:50.282 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:50.277+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:50.291 INFO:teuthology.orchestra.run.smithi122.stdout:891 read 0x1dd5c00 thru 0x1ddebe2 (0x8fe3 bytes) 2024-09-19T21:34:50.292 INFO:teuthology.orchestra.run.smithi122.stdout:892 read 0x29a5a00 thru 0x29a79be (0x1fbf bytes) 2024-09-19T21:34:50.293 INFO:teuthology.orchestra.run.smithi122.stdout:893 write 0x2a0dc00 thru 0x2a12f2f (0x5330 bytes) 2024-09-19T21:34:50.294 INFO:teuthology.orchestra.run.smithi122.stdout:894 read 0x121d600 thru 0x122c8c5 (0xf2c6 bytes) 2024-09-19T21:34:50.295 INFO:teuthology.orchestra.run.smithi122.stdout:895 read 0xe24fe00 thru 0xe255a08 (0x5c09 bytes) 2024-09-19T21:34:50.296 INFO:teuthology.orchestra.run.smithi122.stdout:896 write 0x5f8f000 thru 0x5f96d14 (0x7d15 bytes) 2024-09-19T21:34:50.296 INFO:teuthology.orchestra.run.smithi122.stdout:897 read 0x4956c00 thru 0x4962d69 (0xc16a bytes) 2024-09-19T21:34:50.298 INFO:teuthology.orchestra.run.smithi122.stdout:899 write 0x5e14e00 thru 0x5e1faa5 (0xaca6 bytes) 2024-09-19T21:34:50.299 INFO:teuthology.orchestra.run.smithi122.stdout:901 write 0xaca6600 thru 0xacb5dd5 (0xf7d6 bytes) 2024-09-19T21:34:50.300 INFO:teuthology.orchestra.run.smithi122.stdout:902 read 0x5f2c400 thru 0x5f3624d (0x9e4e bytes) 2024-09-19T21:34:50.301 INFO:teuthology.orchestra.run.smithi122.stdout:905 read 0x7c2e800 thru 0x7c397d0 (0xafd1 bytes) 2024-09-19T21:34:50.302 INFO:teuthology.orchestra.run.smithi122.stdout:907 read 0x513bc00 thru 0x51482f7 (0xc6f8 bytes) 2024-09-19T21:34:50.304 INFO:teuthology.orchestra.run.smithi122.stdout:908 read 0x9efd400 thru 0x9f0a625 (0xd226 bytes) 2024-09-19T21:34:50.305 INFO:teuthology.orchestra.run.smithi122.stdout:910 read 0x46e4c00 thru 0x46e5b4b (0xf4c bytes) 2024-09-19T21:34:50.306 INFO:teuthology.orchestra.run.smithi122.stdout:912 read 0xa8b1400 thru 0xa8b1c5c (0x85d bytes) 2024-09-19T21:34:50.307 INFO:teuthology.orchestra.run.smithi122.stdout:915 write 0xcf1ee00 thru 0xcf1faae (0xcaf bytes) 2024-09-19T21:34:50.307 INFO:teuthology.orchestra.run.smithi122.stdout:917 punch from 0x8c9e00 to 0x8cb400, (0x1600 bytes) 2024-09-19T21:34:50.362 INFO:teuthology.orchestra.run.smithi122.stdout:918 trunc from 0xe35c000 to 0xbf6da00 2024-09-19T21:34:50.382 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:50.377+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:50.386 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1242840520000001, 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:50.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:34:50.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:34:50.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1229575829999998, 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:34:50.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1229261089999998, 2024-09-19T21:34:50.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:50.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1228871550000008, 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1228461119999995, 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.405 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:34:50.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4822 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:50.409 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1136025679999992, 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021034206999999999, 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4822 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:50.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268249+0000", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.316e-06 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268252+0000", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268255+0000", 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9629999999999998e-06 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289149+0000", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.020894302 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289216+0000", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7269999999999998e-05 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289282+0000", 2024-09-19T21:34:50.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5901999999999993e-05 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135937279999997, 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:50.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:34:50.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135834990000006, 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:50.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:34:50.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135740569999992, 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:34:50.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:34:50.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135556169999994, 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:50.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:34:50.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135450089999992, 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:34:50.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:34:50.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135362700000009, 2024-09-19T21:34:50.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:34:50.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:34:50.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1135240209999999, 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:34:50.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1133252070000008, 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:50.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:34:50.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1133073440000008, 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:50.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1132907979999995, 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1132744530000007, 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:34:50.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:34:50.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1132575760000005, 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:34:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1132393890000003, 2024-09-19T21:34:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:34:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.1132208600000002, 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:34:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.479 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.480 INFO:teuthology.orchestra.run.smithi122.stdout:919 read 0x5753400 thru 0x57624b3 (0xf0b4 bytes) 2024-09-19T21:34:50.480 INFO:teuthology.orchestra.run.smithi122.stdout:920 write 0x7b39a00 thru 0x7b3c037 (0x2638 bytes) 2024-09-19T21:34:50.480 INFO:teuthology.orchestra.run.smithi122.stdout:921 punch from 0x92eea00 to 0x92f4a00, (0x6000 bytes) 2024-09-19T21:34:50.481 INFO:teuthology.orchestra.run.smithi122.stdout:922 read 0x7cf3200 thru 0x7cf52f7 (0x20f8 bytes) 2024-09-19T21:34:50.481 INFO:teuthology.orchestra.run.smithi122.stdout:923 write 0xb3b8000 thru 0xb3c333c (0xb33d bytes) 2024-09-19T21:34:50.481 INFO:teuthology.orchestra.run.smithi122.stdout:924 trunc from 0xbf6da00 to 0xa7efa00 2024-09-19T21:34:50.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:50.482 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:50.477+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:50.559 INFO:teuthology.orchestra.run.smithi122.stdout:927 read 0x3d3c000 thru 0x3d4246e (0x646f bytes) 2024-09-19T21:34:50.583 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:50.577+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.584 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:50.584 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:50.584 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.590 INFO:teuthology.orchestra.run.smithi122.stdout:928 write 0xd5d3200 thru 0xd5dce78 (0x9c79 bytes) 2024-09-19T21:34:50.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:50.642 INFO:teuthology.orchestra.run.smithi122.stdout:929 trunc from 0xd5dd000 to 0x44c9200 2024-09-19T21:34:50.684 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:50.677+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:50.692 INFO:teuthology.orchestra.run.smithi122.stdout:931 punch from 0x308ee00 to 0x3099a00, (0xac00 bytes) 2024-09-19T21:34:50.694 INFO:teuthology.orchestra.run.smithi122.stdout:932 punch from 0x23d9200 to 0x23dda00, (0x4800 bytes) 2024-09-19T21:34:50.702 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.703 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:50.703 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:50.703 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:50.703 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.711 INFO:teuthology.orchestra.run.smithi122.stdout:933 trunc from 0x44c9200 to 0xc117000 2024-09-19T21:34:50.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:50.730 INFO:teuthology.orchestra.run.smithi122.stdout:940 read 0x56a8800 thru 0x56b6b5b (0xe35c bytes) 2024-09-19T21:34:50.741 INFO:teuthology.orchestra.run.smithi122.stdout:941 trunc from 0xc117000 to 0x9bb0400 2024-09-19T21:34:50.764 INFO:teuthology.orchestra.run.smithi122.stdout:942 punch from 0x2d23a00 to 0x2d25800, (0x1e00 bytes) 2024-09-19T21:34:50.784 INFO:teuthology.orchestra.run.smithi122.stdout:943 write 0xaaf5200 thru 0xaafa2d9 (0x50da bytes) 2024-09-19T21:34:50.785 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:50.781+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:50.787 INFO:teuthology.orchestra.run.smithi122.stdout:946 read 0xf7c400 thru 0xf7eb55 (0x2756 bytes) 2024-09-19T21:34:50.788 INFO:teuthology.orchestra.run.smithi122.stdout:948 punch from 0x9f96200 to 0x9fa5600, (0xf400 bytes) 2024-09-19T21:34:50.789 INFO:teuthology.orchestra.run.smithi122.stdout:951 read 0xb58200 thru 0xb5d1f1 (0x4ff2 bytes) 2024-09-19T21:34:50.790 INFO:teuthology.orchestra.run.smithi122.stdout:952 read 0x836f000 thru 0x837230b (0x330c bytes) 2024-09-19T21:34:50.809 INFO:teuthology.orchestra.run.smithi122.stdout:954 trunc from 0xaafa400 to 0x471dc00 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.826 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 34.887477611000001, 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:34:50.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:34:50.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4788 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5512518820000007, 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053208699999999998, 2024-09-19T21:34:50.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4788 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269024+0000", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269027+0000", 2024-09-19T21:34:50.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0889999999999998e-06 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269032+0000", 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.31e-06 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269050+0000", 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8138000000000002e-05 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322058+0000", 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053008161999999998 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322126+0000", 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8269999999999995e-05 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322233+0000", 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107027 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.837 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4789 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 423936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5512213100000007, 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053201143999999999, 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4789 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269066+0000", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1392e-05 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269073+0000", 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0500000000000003e-06 2024-09-19T21:34:50.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269079+0000", 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322103+0000", 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053024486000000003 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322168+0000", 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949000000000004e-05 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322256+0000", 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8187000000000001e-05 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.842 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4790 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5511934820000004, 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053307251999999999, 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4790 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269097+0000", 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4481000000000001e-05 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0243e-05 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0139999999999998e-06 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322255+0000", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053142366000000003 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322323+0000", 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7725999999999995e-05 2024-09-19T21:34:50.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322390+0000", 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7800999999999997e-05 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4791 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5511595979999999, 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053280393000000002, 2024-09-19T21:34:50.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4791 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269119+0000", 2024-09-19T21:34:50.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269125+0000", 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5180000000000004e-06 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269129+0000", 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6990000000000002e-06 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322279+0000", 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053149890999999998 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322342+0000", 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2392999999999997e-05 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322397+0000", 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5798999999999999e-05 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4792 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5511436360000008, 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053414182999999997, 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4792 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.852 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269141+0000", 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6810000000000003e-06 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3549999999999998e-06 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322416+0000", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053270709999999999 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322474+0000", 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8161000000000001e-05 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322547+0000", 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3357999999999998e-05 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4796 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5511279879999993, 2024-09-19T21:34:50.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053558802000000003, 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4796 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269148+0000", 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0439999999999998e-06 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269155+0000", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.967e-06 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269167+0000", 2024-09-19T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1935e-05 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322574+0000", 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053407102999999997 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322642+0000", 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7683999999999998e-05 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322707+0000", 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5421999999999998e-05 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4799 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5511049890000006, 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054302238000000003, 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4799 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269171+0000", 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0429999999999999e-06 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269186+0000", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9840000000000001e-06 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323330+0000", 2024-09-19T21:34:50.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054144172999999997 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323399+0000", 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9543999999999997e-05 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323474+0000", 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4564e-05 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.865 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4868 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5510864340000001, 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053685682999999998, 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4868 2024-09-19T21:34:50.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269192+0000", 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2639999999999998e-06 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269198+0000", 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8810000000000001e-06 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1049999999999997e-06 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322728+0000", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053525831000000003 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322792+0000", 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4367000000000002e-05 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322876+0000", 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3558000000000002e-05 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4869 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5510703909999997, 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053842947000000002, 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4869 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269208+0000", 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3879999999999998e-06 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269214+0000", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269221+0000", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.657e-06 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322897+0000", 2024-09-19T21:34:50.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053676662 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322966+0000", 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9085999999999996e-05 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323049+0000", 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2742000000000002e-05 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4870 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5508666190000007, 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053808917999999997, 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4870 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269225+0000", 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998155 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269413+0000", 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018773800000000001 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269420+0000", 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7399999999999998e-06 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269424+0000", 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9389999999999999e-06 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323071+0000", 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053647087000000003 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323137+0000", 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5810999999999997e-05 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323219+0000", 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2236000000000003e-05 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4871 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5502256429999992, 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053346843999999997, 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4871 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269436+0000", 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9993844 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270057+0000", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062110799999999999 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270066+0000", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2709999999999998e-06 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270073+0000", 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9770000000000003e-06 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323240+0000", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053167430000000002 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323312+0000", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1581000000000005e-05 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323398+0000", 2024-09-19T21:34:50.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5939999999999994e-05 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4872 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5501971010000002, 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053465749, 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4872 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270079+0000", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270082+0000", 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9730000000000002e-06 2024-09-19T21:34:50.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270088+0000", 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270093+0000", 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323425+0000", 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053332349000000001 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323493+0000", 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7828000000000002e-05 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323545+0000", 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2166000000000002e-05 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.888 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4873 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 106496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5501792880000007, 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053689477999999999, 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4873 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270099+0000", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270106+0000", 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:34:50.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270111+0000", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7520000000000004e-06 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323583+0000", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053471779999999997 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323664+0000", 2024-09-19T21:34:50.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1402000000000004e-05 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323787+0000", 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122404 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4874 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 116736~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5501522130000005, 2024-09-19T21:34:50.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053900483999999999, 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4874 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270131+0000", 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9079999999999998e-06 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270138+0000", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5699999999999998e-06 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270143+0000", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323826+0000", 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053683467999999998 2024-09-19T21:34:50.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323902+0000", 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5350000000000002e-05 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324025+0000", 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012334600000000001 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4875 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5501164500000009, 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054010483999999997, 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4875 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270177+0000", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6985000000000001e-05 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270188+0000", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0993e-05 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270193+0000", 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7679999999999997e-06 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324068+0000", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053874248 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324119+0000", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1615e-05 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324171+0000", 2024-09-19T21:34:50.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440999999999998e-05 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4876 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5498320620000001, 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053834246000000002, 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4876 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270198+0000", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997535 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270450+0000", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025216000000000003 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270460+0000", 2024-09-19T21:34:50.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8409999999999995e-06 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270467+0000", 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6680000000000004e-06 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324197+0000", 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053730214999999998 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324239+0000", 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1755999999999998e-05 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324279+0000", 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0224e-05 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.906 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4877 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5498027840000006, 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054028259000000002, 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4877 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270473+0000", 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270476+0000", 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8590000000000001e-06 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270490+0000", 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4071e-05 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270496+0000", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7599999999999999e-06 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324308+0000", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053811953000000003 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324349+0000", 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0747999999999997e-05 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324502+0000", 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015331599999999999 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4878 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5497554359999999, 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054078566000000002, 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4878 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:50.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270524+0000", 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271662+0000", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001138114 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271683+0000", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0638000000000001e-05 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324520+0000", 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052837566000000002 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324559+0000", 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8782999999999998e-05 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324600+0000", 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.066e-05 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.915 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4881 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.5482281540000002, 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053152144999999998, 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4881 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:50.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272051+0000", 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8890000000000002e-06 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272057+0000", 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6030000000000004e-06 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:50.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272066+0000", 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5540000000000003e-06 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325024+0000", 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052957551999999998 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325114+0000", 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0708e-05 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:50.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325201+0000", 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6236999999999994e-05 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:50.920 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:50.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:50.923 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:50.917+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:50.923 INFO:teuthology.orchestra.run.smithi122.stdout:956 punch from 0x1e06e00 to 0x1e0fe00, (0x9000 bytes) 2024-09-19T21:34:50.923 INFO:teuthology.orchestra.run.smithi122.stdout:957 punch from 0x2c07e00 to 0x2c15a00, (0xdc00 bytes) 2024-09-19T21:34:50.923 INFO:teuthology.orchestra.run.smithi122.stdout:964 trunc from 0x471dc00 to 0xd24e400 2024-09-19T21:34:50.923 INFO:teuthology.orchestra.run.smithi122.stdout:969 read 0x16b7600 thru 0x16c1c9a (0xa69b bytes) 2024-09-19T21:34:50.923 INFO:teuthology.orchestra.run.smithi122.stdout:970 write 0xc608600 thru 0xc617e8b (0xf88c bytes) 2024-09-19T21:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:971 write 0xd842c00 thru 0xd84a96b (0x7d6c bytes) 2024-09-19T21:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:973 write 0x4083200 thru 0x408d6c6 (0xa4c7 bytes) 2024-09-19T21:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:975 read 0x1f3cc00 thru 0x1f3fdcb (0x31cc bytes) 2024-09-19T21:34:50.924 INFO:teuthology.orchestra.run.smithi122.stdout:977 trunc from 0xd84aa00 to 0xae7e600 2024-09-19T21:34:50.960 INFO:teuthology.orchestra.run.smithi122.stdout:979 read 0x4d9d600 thru 0x4dacd36 (0xf737 bytes) 2024-09-19T21:34:50.997 INFO:teuthology.orchestra.run.smithi122.stdout:983 write 0xee68000 thru 0xee6b222 (0x3223 bytes) 2024-09-19T21:34:50.999 INFO:teuthology.orchestra.run.smithi122.stdout:984 write 0xb71f000 thru 0xb71fa10 (0xa11 bytes) 2024-09-19T21:34:50.999 INFO:teuthology.orchestra.run.smithi122.stdout:988 read 0xde42e00 thru 0xde52d35 (0xff36 bytes) 2024-09-19T21:34:51.022 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:51.017+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:51.046 INFO:teuthology.orchestra.run.smithi122.stdout:992 trunc from 0xee6b400 to 0x280e00 2024-09-19T21:34:51.082 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:51.082 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:51.082 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:51.082 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:51.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:51.115 INFO:teuthology.orchestra.run.smithi122.stdout:993 read 0xbd400 thru 0xc9ad8 (0xc6d9 bytes) 2024-09-19T21:34:51.123 INFO:teuthology.orchestra.run.smithi122.stdout:995 punch from 0x25600 to 0x2be00, (0x6800 bytes) 2024-09-19T21:34:51.123 INFO:teuthology.orchestra.run.smithi122.stdout:999 punch from 0x179c00 to 0x17ae00, (0x1200 bytes) 2024-09-19T21:34:51.124 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:51.117+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:51.133 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:34:51.154 INFO:teuthology.orchestra.run.smithi122.stdout:1005 write 0x8b37e00 thru 0x8b46a9a (0xec9b bytes) 2024-09-19T21:34:51.157 INFO:teuthology.orchestra.run.smithi122.stdout:1006 read 0x1b86e00 thru 0x1b9620c (0xf40d bytes) 2024-09-19T21:34:51.160 INFO:teuthology.orchestra.run.smithi122.stdout:1007 write 0x224fc00 thru 0x2251e92 (0x2293 bytes) 2024-09-19T21:34:51.161 INFO:teuthology.orchestra.run.smithi122.stdout:1008 punch from 0x7e51400 to 0x7e60400, (0xf000 bytes) 2024-09-19T21:34:51.162 INFO:teuthology.orchestra.run.smithi122.stdout:1009 read 0x6d40400 thru 0x6d40d2e (0x92f bytes) 2024-09-19T21:34:51.168 INFO:teuthology.orchestra.run.smithi122.stdout:1011 write 0x42c1000 thru 0x42d0b26 (0xfb27 bytes) 2024-09-19T21:34:51.175 INFO:teuthology.orchestra.run.smithi122.stdout:1012 trunc from 0x8b46c00 to 0xaccfe00 2024-09-19T21:34:51.197 INFO:teuthology.orchestra.run.smithi122.stdout:1015 read 0x633fc00 thru 0x634693d (0x6d3e bytes) 2024-09-19T21:34:51.199 INFO:teuthology.orchestra.run.smithi122.stdout:1016 punch from 0x8366800 to 0x836d000, (0x6800 bytes) 2024-09-19T21:34:51.200 INFO:teuthology.orchestra.run.smithi122.stdout:1018 write 0x76db000 thru 0x76df988 (0x4989 bytes) 2024-09-19T21:34:51.207 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:51.207 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:51.207 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:51.207 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:51.207 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:51.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:51.224 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:51.221+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:51.227 INFO:teuthology.orchestra.run.smithi122.stdout:1019 write 0xe6a8400 thru 0xe6b07e9 (0x83ea bytes) 2024-09-19T21:34:51.230 INFO:teuthology.orchestra.run.smithi122.stdout:1020 read 0xbe6bc00 thru 0xbe6e554 (0x2955 bytes) 2024-09-19T21:34:51.230 INFO:teuthology.orchestra.run.smithi122.stdout:1022 read 0xb21600 thru 0xb2b0e3 (0x9ae4 bytes) 2024-09-19T21:34:51.252 INFO:teuthology.orchestra.run.smithi122.stdout:1023 trunc from 0xe6b0800 to 0x56d3a00 2024-09-19T21:34:51.325 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:51.321+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:51.333 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.101112724, 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.334 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:51.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:34:51.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:592 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 2162688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.174039+0000", 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.15451466, 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017056327999999999, 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 592 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174039+0000", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174039+0000", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174038+0000", 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174043+0000", 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1429999999999997e-06 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174056+0000", 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2986000000000001e-05 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.174067+0000", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0220999999999999e-05 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.182477+0000", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0084107839999999993 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.183566+0000", 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001088297 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.191095+0000", 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0075297000000000003 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1162 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180460+0000", 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1480935219999999, 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016215625000000001, 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1162 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180460+0000", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180460+0000", 2024-09-19T21:34:51.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180460+0000", 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180462+0000", 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180468+0000", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6779999999999997e-06 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180473+0000", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9450000000000001e-06 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196577+0000", 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016103754000000001 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196621+0000", 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4066000000000001e-05 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196676+0000", 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5133999999999998e-05 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1163 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 13893632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180477+0000", 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1480765150000001, 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016314927, 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1163 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180477+0000", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180477+0000", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180477+0000", 2024-09-19T21:34:51.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180479+0000", 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180485+0000", 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7769999999999999e-06 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180490+0000", 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1619999999999999e-06 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196699+0000", 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.01620926 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196742+0000", 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2242e-05 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196792+0000", 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0544999999999998e-05 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1164 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 16187392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180494+0000", 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1480594580000001, 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016410397, 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1164 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180494+0000", 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180494+0000", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180494+0000", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180496+0000", 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3949999999999999e-06 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180502+0000", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6919999999999999e-06 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180508+0000", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1789999999999996e-06 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196816+0000", 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016307952000000001 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196851+0000", 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4697999999999999e-05 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196905+0000", 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3826999999999998e-05 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1165 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180512+0000", 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.148041396, 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016537755000000001, 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1165 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180512+0000", 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180512+0000", 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180512+0000", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180514+0000", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180520+0000", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8449999999999999e-06 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180526+0000", 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8320000000000002e-06 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196929+0000", 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016403302000000002 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.196992+0000", 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2236000000000005e-05 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197050+0000", 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8461999999999997e-05 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1166 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180530+0000", 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.148023456, 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016627492000000001, 2024-09-19T21:34:51.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1166 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180530+0000", 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180530+0000", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180530+0000", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180532+0000", 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.463e-06 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180553+0000", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0406000000000001e-05 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180559+0000", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7799999999999997e-06 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197074+0000", 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016515003 2024-09-19T21:34:51.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197111+0000", 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7735999999999999e-05 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197158+0000", 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6482e-05 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1193 3.1d 3:bb5e54f5:::rbd_data.1106ee579bc7.0000000000000002:8 [sparse-read 589824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180948+0000", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1476063059999999, 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016436447, 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1193 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180948+0000", 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180948+0000", 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180946+0000", 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180952+0000", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6910000000000002e-06 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180964+0000", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2208000000000001e-05 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180972+0000", 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4479999999999997e-06 2024-09-19T21:34:51.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197294+0000", 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016322673999999999 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197335+0000", 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0156e-05 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197384+0000", 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9302000000000003e-05 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1195 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.180978+0000", 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1475753230000001, 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016292983, 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.370 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1195 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180978+0000", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180978+0000", 2024-09-19T21:34:51.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180978+0000", 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180981+0000", 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2540000000000001e-06 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180988+0000", 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5300000000000002e-06 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.180993+0000", 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4689999999999996e-06 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197179+0000", 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016185701 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197222+0000", 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2784999999999998e-05 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197271+0000", 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9913999999999997e-05 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1202 3.1d 3:bb5e54f5:::rbd_data.1106ee579bc7.0000000000000002:8 [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.374 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.181025+0000", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.147529244, 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016601517999999999, 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1202 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181025+0000", 2024-09-19T21:34:51.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181025+0000", 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181024+0000", 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181027+0000", 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8109999999999999e-06 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181033+0000", 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1589999999999998e-06 2024-09-19T21:34:51.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181039+0000", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4210000000000003e-06 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197517+0000", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016478080999999999 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197555+0000", 2024-09-19T21:34:51.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8423999999999997e-05 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197626+0000", 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1045999999999997e-05 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1203 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.181043+0000", 2024-09-19T21:34:51.379 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.147510628, 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016450270999999999, 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1203 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181043+0000", 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181043+0000", 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181043+0000", 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181051+0000", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.892e-06 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181057+0000", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4790000000000001e-06 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181063+0000", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2829999999999998e-06 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197404+0000", 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016340606000000001 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197445+0000", 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1355e-05 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197493+0000", 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8053000000000002e-05 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1210 3.1d 3:bb5e54f5:::rbd_data.1106ee579bc7.0000000000000002:8 [sparse-read 2031616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.181087+0000", 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1474670579999999, 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016763047999999999, 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1210 2024-09-19T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181087+0000", 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181087+0000", 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181086+0000", 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181097+0000", 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0855e-05 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181103+0000", 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6499999999999999e-06 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181109+0000", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1549999999999998e-06 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197762+0000", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016653127 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197800+0000", 2024-09-19T21:34:51.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8729000000000001e-05 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197850+0000", 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9305999999999999e-05 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1212 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.181114+0000", 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1474397700000001, 2024-09-19T21:34:51.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016624501, 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1212 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181114+0000", 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181114+0000", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181113+0000", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181116+0000", 2024-09-19T21:34:51.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.864e-06 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181122+0000", 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0669999999999997e-06 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181127+0000", 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7709999999999997e-06 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197646+0000", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016519191999999999 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197688+0000", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2005000000000002e-05 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197739+0000", 2024-09-19T21:34:51.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0194000000000001e-05 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:1220 3.1d 3:bb5e54f5:::rbd_data.1106ee579bc7.0000000000000002:8 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.181132+0000", 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.147422197, 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016829047, 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1220 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181132+0000", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181132+0000", 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181131+0000", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181134+0000", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181139+0000", 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:34:51.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.181144+0000", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197872+0000", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016728684000000001 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197912+0000", 2024-09-19T21:34:51.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9538e-05 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.197961+0000", 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8773000000000002e-05 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2593 3.1d 3:ba48e33c:::rbd_data.1106e9705673.0000000000000000:9 [sparse-read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.463893+0000", 2024-09-19T21:34:51.397 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86466033600000003, 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018101478000000001, 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2593 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463893+0000", 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463893+0000", 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463893+0000", 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463896+0000", 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9660000000000001e-06 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463902+0000", 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7320000000000003e-06 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.463907+0000", 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8899999999999998e-06 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.475923+0000", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012016551 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.475991+0000", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7409999999999993e-05 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481995+0000", 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0060043040000000002 2024-09-19T21:34:51.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2672 3.1d 3:bd6b9829:::rbd_data.1106ee579bc7.0000000000000003:8 [sparse-read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.464392+0000", 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86416156099999997, 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016612025999999998, 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2672 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464392+0000", 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464392+0000", 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464392+0000", 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464394+0000", 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2819999999999999e-06 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464399+0000", 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2660000000000001e-06 2024-09-19T21:34:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464404+0000", 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6460000000000001e-06 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.480810+0000", 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016405669000000001 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.480900+0000", 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9884999999999997e-05 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481004+0000", 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010471000000000001 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2688 3.1d 3:bd6b9829:::rbd_data.1106ee579bc7.0000000000000003:8 [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.406 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.464423+0000", 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.864130701, 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016826503999999999, 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2688 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464423+0000", 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464423+0000", 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464423+0000", 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464425+0000", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1040000000000001e-06 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464430+0000", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5829999999999997e-06 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464437+0000", 2024-09-19T21:34:51.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1920000000000002e-06 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481035+0000", 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016598182999999999 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481141+0000", 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000105717 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481250+0000", 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010902400000000001 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2705 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 3080192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.464479+0000", 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86407432799999995, 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017060825000000002, 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2705 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464479+0000", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464479+0000", 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464479+0000", 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464484+0000", 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3529999999999997e-06 2024-09-19T21:34:51.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464489+0000", 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9490000000000001e-06 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464494+0000", 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.702e-06 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481276+0000", 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.016782115 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481423+0000", 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014668600000000001 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481540+0000", 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011733199999999999 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2719 3.1d 3:bb5e54f5:::rbd_data.1106ee579bc7.0000000000000002:8 [sparse-read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.464520+0000", 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86403361400000001, 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017245023000000002, 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2719 2024-09-19T21:34:51.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464520+0000", 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464520+0000", 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464520+0000", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464523+0000", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1709999999999998e-06 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464529+0000", 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464534+0000", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0390000000000004e-06 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481567+0000", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017033303999999999 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481663+0000", 2024-09-19T21:34:51.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6161000000000003e-05 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481765+0000", 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000101804 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4720.0:2721 3.5 3:a6f37d37:::rbd_data.1106ee579bc7.0000000000000000:8 [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:51.420 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:50.464538+0000", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86401554300000005, 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017447863000000001, 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4720", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2721 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464538+0000", 2024-09-19T21:34:51.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464538+0000", 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464538+0000", 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:51.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464540+0000", 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464547+0000", 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8249999999999999e-06 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.464553+0000", 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5770000000000001e-06 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481791+0000", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.017238834000000001 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481885+0000", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3602000000000003e-05 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:50.481986+0000", 2024-09-19T21:34:51.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000101053 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:51.425 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:51.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:51.428 INFO:teuthology.orchestra.run.smithi122.stdout:1024 write 0xbdbd000 thru 0xbdc7ddd (0xadde bytes) 2024-09-19T21:34:51.428 INFO:teuthology.orchestra.run.smithi122.stdout:1025 write 0xb0c7e00 thru 0xb0d28e9 (0xaaea bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1028 punch from 0xbc1c400 to 0xbc20200, (0x3e00 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1029 write 0x7e50800 thru 0x7e5acd7 (0xa4d8 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1030 read 0xab08e00 thru 0xab149d4 (0xbbd5 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1031 punch from 0x6ce2a00 to 0x6ceb400, (0x8a00 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1032 read 0x1ae5400 thru 0x1aed48c (0x808d bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1033 trunc from 0xbdc7e00 to 0xb2dd000 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1034 read 0x55be00 thru 0x55e877 (0x2a78 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1035 write 0x81f5200 thru 0x8202bd4 (0xd9d5 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1041 write 0xc7a8200 thru 0xc7acb2c (0x492d bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1042 write 0x4e3c400 thru 0x4e497d2 (0xd3d3 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1044 write 0x1536e00 thru 0x153b52f (0x4730 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1046 read 0xa0a8800 thru 0xa0b2593 (0x9d94 bytes) 2024-09-19T21:34:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:1049 write 0x3516a00 thru 0x3520e56 (0xa457 bytes) 2024-09-19T21:34:51.430 INFO:teuthology.orchestra.run.smithi122.stdout:1051 flatten 2024-09-19T21:34:51.430 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:51.421+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:51.439 INFO:teuthology.orchestra.run.smithi122.stdout:1053 write 0x1b25400 thru 0x1b34ceb (0xf8ec bytes) 2024-09-19T21:34:51.442 INFO:teuthology.orchestra.run.smithi122.stdout:1054 read 0xa39f800 thru 0xa3a0015 (0x816 bytes) 2024-09-19T21:34:51.443 INFO:teuthology.orchestra.run.smithi122.stdout:1055 read 0x9a64200 thru 0x9a6753d (0x333e bytes) 2024-09-19T21:34:51.444 INFO:teuthology.orchestra.run.smithi122.stdout:1058 write 0xb061400 thru 0xb06d721 (0xc322 bytes) 2024-09-19T21:34:51.445 INFO:teuthology.orchestra.run.smithi122.stdout:1059 write 0x7843400 thru 0x7852ef9 (0xfafa bytes) 2024-09-19T21:34:51.446 INFO:teuthology.orchestra.run.smithi122.stdout:1061 write 0x73a5e00 thru 0x73a9d5f (0x3f60 bytes) 2024-09-19T21:34:51.447 INFO:teuthology.orchestra.run.smithi122.stdout:1063 write 0x3922800 thru 0x3929d77 (0x7578 bytes) 2024-09-19T21:34:51.448 INFO:teuthology.orchestra.run.smithi122.stdout:1064 read 0xc562c00 thru 0xc569976 (0x6d77 bytes) 2024-09-19T21:34:51.449 INFO:teuthology.orchestra.run.smithi122.stdout:1065 punch from 0x9c90600 to 0x9c92000, (0x1a00 bytes) 2024-09-19T21:34:51.451 INFO:teuthology.orchestra.run.smithi122.stdout:1066 read 0x2071c00 thru 0x20767c1 (0x4bc2 bytes) 2024-09-19T21:34:51.455 INFO:teuthology.orchestra.run.smithi122.stdout:1068 read 0xc1fa400 thru 0xc20502f (0xac30 bytes) 2024-09-19T21:34:51.456 INFO:teuthology.orchestra.run.smithi122.stdout:1069 read 0x8004e00 thru 0x80128e8 (0xdae9 bytes) 2024-09-19T21:34:51.457 INFO:teuthology.orchestra.run.smithi122.stdout:1071 clone 7 order 18 su 32768 sc 6 2024-09-19T21:34:51.528 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:51.521+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:51.545 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:51.545 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:51.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:51.629 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:51.625+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:51.659 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:51.659 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:51.659 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:51.659 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:51.659 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:51.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:51.729 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:51.725+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:51.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:51.780 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4296 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523123839, 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022203509999999999, 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4296 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253833+0000", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.213e-06 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253840+0000", 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5930000000000002e-06 2024-09-19T21:34:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253845+0000", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3979999999999998e-06 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275901+0000", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022056172999999998 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275964+0000", 2024-09-19T21:34:51.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2723000000000002e-05 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276034+0000", 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9774e-05 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4297 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:51.785 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523103979, 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359332999999999, 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4297 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:51.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253852+0000", 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4779999999999998e-06 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253858+0000", 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7740000000000001e-06 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253860+0000", 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276057+0000", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022196344999999999 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276120+0000", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3521000000000004e-05 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276209+0000", 2024-09-19T21:34:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9135999999999995e-05 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4298 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523089429000001, 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022554268999999998, 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4298 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253864+0000", 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253873+0000", 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0289999999999994e-06 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253880+0000", 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4729999999999997e-06 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253888+0000", 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276235+0000", 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022346462000000001 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276323+0000", 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7990000000000003e-05 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276419+0000", 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6291000000000004e-05 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4299 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 335872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:51.794 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523051831, 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022708860000000001, 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4299 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253905+0000", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.591e-06 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253921+0000", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5282e-05 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253927+0000", 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.003e-06 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276468+0000", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022541017 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276535+0000", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7119000000000006e-05 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276611+0000", 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6347000000000006e-05 2024-09-19T21:34:51.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4300 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523024147999999, 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022876461000000001, 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4300 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253932+0000", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.857e-06 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253935+0000", 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:34:51.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253938+0000", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.441e-06 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276636+0000", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022698381 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276702+0000", 2024-09-19T21:34:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5258000000000004e-05 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276806+0000", 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010488399999999999 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4301 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.523014149, 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023114603000000001, 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4301 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:51.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253941+0000", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.44e-06 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253944+0000", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0960000000000001e-06 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253947+0000", 2024-09-19T21:34:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4109999999999999e-06 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276839+0000", 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022892550000000001 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276947+0000", 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107793 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277055+0000", 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010752299999999999 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4302 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.522780187, 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023063824, 2024-09-19T21:34:51.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4302 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254173+0000", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254179+0000", 2024-09-19T21:34:51.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0789999999999997e-06 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254190+0000", 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0532e-05 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254196+0000", 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6560000000000003e-06 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277079+0000", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022882731 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277146+0000", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6934000000000007e-05 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277238+0000", 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1891999999999994e-05 2024-09-19T21:34:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4303 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.522751613000001, 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023222402, 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4303 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:51.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254205+0000", 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8329999999999998e-06 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254212+0000", 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6549999999999998e-06 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254217+0000", 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277263+0000", 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023045955 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277333+0000", 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0257000000000002e-05 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277425+0000", 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1952000000000003e-05 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4304 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.522708528000001, 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023378263, 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4304 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254245+0000", 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254248+0000", 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2940000000000001e-06 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254256+0000", 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254261+0000", 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5310000000000001e-06 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277449+0000", 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023187433 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277538+0000", 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9474000000000004e-05 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277624+0000", 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5560000000000001e-05 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4305 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 446464~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.522683709000001, 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033521316000000002, 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4305 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254273+0000", 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0850000000000001e-06 2024-09-19T21:34:51.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254280+0000", 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0720000000000002e-06 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254285+0000", 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3839999999999996e-06 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277648+0000", 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0233627 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277727+0000", 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9128999999999994e-05 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.287792+0000", 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010064559000000001 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.826 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4307 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.522503178999999, 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027454658999999999, 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4307 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254450+0000", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254455+0000", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4560000000000002e-06 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254464+0000", 2024-09-19T21:34:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8570000000000007e-06 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254471+0000", 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281760+0000", 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027288818999999999 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281834+0000", 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4019999999999999e-05 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281906+0000", 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1353999999999995e-05 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4309 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.521924330000001, 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027050399999999999, 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4309 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255029+0000", 2024-09-19T21:34:51.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255035+0000", 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6500000000000001e-06 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255046+0000", 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1382999999999999e-05 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255061+0000", 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.554e-05 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281929+0000", 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02686796 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282019+0000", 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9860999999999999e-05 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282080+0000", 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0891999999999999e-05 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4311 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.521742193, 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027091388000000001, 2024-09-19T21:34:51.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4311 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255211+0000", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255216+0000", 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3319999999999999e-06 2024-09-19T21:34:51.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255228+0000", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1202999999999999e-05 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255235+0000", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0350000000000004e-06 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282101+0000", 2024-09-19T21:34:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026866476 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282202+0000", 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282303+0000", 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100883 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4313 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.521703218000001, 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027229189000000001, 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4313 2024-09-19T21:34:51.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255250+0000", 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255254+0000", 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0629999999999999e-06 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255262+0000", 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-06 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255269+0000", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4050000000000003e-06 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282326+0000", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027057184000000001 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282403+0000", 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6720999999999999e-05 2024-09-19T21:34:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282480+0000", 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7001000000000004e-05 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4744 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:51.845 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507864529000001, 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022372296999999999, 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4744 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269089+0000", 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269092+0000", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269102+0000", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0077e-05 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0289999999999996e-06 2024-09-19T21:34:51.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291277+0000", 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022170091999999999 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291356+0000", 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9241999999999998e-05 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291462+0000", 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105609 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4746 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507842825999999, 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022538171999999999, 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4746 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269123+0000", 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0820000000000005e-06 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269128+0000", 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0549999999999999e-06 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291488+0000", 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359917 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291558+0000", 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0339000000000005e-05 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291649+0000", 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1558000000000007e-05 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4747 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507821179, 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022815518999999999, 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4747 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0185e-05 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:51.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7169999999999999e-06 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291712+0000", 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022561748 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291806+0000", 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3826999999999995e-05 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291948+0000", 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014276600000000001 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4749 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507799931999999, 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023058227000000001, 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4749 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269156+0000", 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2230000000000001e-06 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269165+0000", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8470000000000002e-06 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269170+0000", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8489999999999997e-06 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291977+0000", 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022806918999999998 2024-09-19T21:34:51.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1191000000000003e-05 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292212+0000", 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016450899999999999 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4751 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507780057, 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023246788000000001, 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4751 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6450000000000005e-06 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269191+0000", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9550000000000004e-06 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269196+0000", 2024-09-19T21:34:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8500000000000002e-06 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292239+0000", 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023042450999999999 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292360+0000", 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121804 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292421+0000", 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0467000000000002e-05 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4753 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.507752201000001, 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023417911999999999, 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:51.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4753 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269204+0000", 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.481e-06 2024-09-19T21:34:51.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269213+0000", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000005e-06 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269219+0000", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1919999999999996e-06 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292445+0000", 2024-09-19T21:34:51.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023226009999999998 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292517+0000", 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1898999999999997e-05 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292620+0000", 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103198 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:51.873 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:51.873 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:51.874 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:51.875 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:51.869+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:51.921 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:51.944 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:51.969 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:51.975 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:51.969+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:51.989 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:51.989 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:51.989 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:51.989 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:51.990 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:51.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:52.013 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:52.035 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:52.038 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:52.076 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:52.069+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:52.106 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.106 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:52.177 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:52.173+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:52.230 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.230 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:52.288 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:52.285+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:52.348 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.348 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:52.380 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:52.373+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:52.468 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.468 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:52.481 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:52.477+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:52.581 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:52.577+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:52.582 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.582 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:52.683 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:52.677+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:52.711 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.711 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.721 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:52.784 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:52.781+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:52.827 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.827 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:52.885 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:52.881+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:52.943 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:52.943 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:52.953 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:52.954 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-19T21:34:52.986 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:52.981+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:53.081 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.081 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.086 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:53.081+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:53.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:53.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:34:53.193 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:53.189+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:53.220 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.220 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:53.289 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:53.285+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:53.335 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.335 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:53.355 INFO:teuthology.orchestra.run.smithi122.stdout:1074 trunc from 0xc7acc00 to 0x2993600 2024-09-19T21:34:53.389 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:53.385+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:53.399 INFO:teuthology.orchestra.run.smithi122.stdout:1075 clone 8 order 25 su 16777216 sc 3 2024-09-19T21:34:53.456 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.456 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:53.490 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:53.485+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:53.571 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.571 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:53.625 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:53.621+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:53.692 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:53.689+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:53.735 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.735 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:53.793 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:53.789+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:53.862 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.863 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:53.894 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:53.889+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:53.913 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:34:53.914 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:34:53.914 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T21:34:53.914 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:34:53.914 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:34:53.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T21:34:53.976 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:53.976 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:53.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:53.995 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:53.989+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:54.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:54.096 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:54.089+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:54.096 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:54.096 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:54.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:54.143 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:34:54.196 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:54.189+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:54.198 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:54.216 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:54.216 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:54.220 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:54.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:54.243 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:54.265 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:54.268 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone7 from 0x2993600 (overlap 0x2993600) to 0x1d1a400 2024-09-19T21:34:54.298 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:54.293+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:54.298 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:54.325 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:54.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:54.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:54.354 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:54.354 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:54.354 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:54.354 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:54.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:54.397 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:54.393+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:54.498 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:54.493+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.124034784999999, 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:34:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.124014477999999, 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:34:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:34:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.123995015999999, 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.123960705000002, 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:34:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.123935204999999, 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119981975999998, 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:34:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119944852, 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:34:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:34:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119907633, 2024-09-19T21:34:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119863244000001, 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119845557000001, 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:34:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119820607000001, 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:34:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119781316000001, 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:34:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119743913000001, 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:34:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.119726084, 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:34:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.116077306000001, 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:34:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.116052761999999, 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:34:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.116034147000001, 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:34:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:34:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.115984740999998, 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:34:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:34:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.115966343, 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.618 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.115940574, 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:34:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:54.623 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:54.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:54.625 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:54.621+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:54.726 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:54.721+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:54.741 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:54.741 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:54.742 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:54.742 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:54.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:54.827 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:54.821+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:54.902 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-19T21:34:54.910 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:54.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:54.910 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:54.910 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:54.910 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:54.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:54.927 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:54.921+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:55.028 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:55.025+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3027 3.1f 3:f973b0ba:::rbd_data.11064af7243d.000000000000000d:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.811458995000001, 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014783325, 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3027 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231965+0000", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231962+0000", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999976 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.231977+0000", 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4780000000000001e-05 2024-09-19T21:34:55.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232006+0000", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9094e-05 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232025+0000", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8771e-05 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232146+0000", 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120834 2024-09-19T21:34:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232377+0000", 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023080400000000001 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.232837+0000", 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00046002400000000001 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.243127+0000", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010290509999999999 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246633+0000", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0035057460000000001 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246695+0000", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1860999999999996e-05 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246748+0000", 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3476000000000001e-05 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3065 3.1f 3:fb335e39:::rbd_data.11064af7243d.0000000000000021:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.810372159, 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014219874, 2024-09-19T21:34:55.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3065 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233052+0000", 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233050+0000", 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233063+0000", 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2850999999999999e-05 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233086+0000", 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3125999999999998e-05 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233106+0000", 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0273000000000001e-05 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236241+0000", 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031351590000000002 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236353+0000", 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011176899999999999 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.236627+0000", 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027411599999999998 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.246167+0000", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0095400050000000007 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247178+0000", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001010628 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247224+0000", 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6786000000000002e-05 2024-09-19T21:34:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247272+0000", 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7264e-05 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:3075 3.1f 3:ffdae854:::rbd_data.11064af7243d.000000000000009b:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 13.810299741, 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015477372, 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 3075 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233124+0000", 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233123+0000", 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233134+0000", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.046e-05 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233162+0000", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8530999999999998e-05 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.233177+0000", 2024-09-19T21:34:55.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4894e-05 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239745+0000", 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0065679969999999999 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.239854+0000", 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108999 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:34:55.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.240035+0000", 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018081799999999999 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.247400+0000", 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0073650909999999998 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248505+0000", 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0011052779999999999 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248555+0000", 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0175e-05 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.248602+0000", 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6343e-05 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:1358 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14876672~1024,write 16591360~25600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.210461+0000", 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8329630420000003, 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.015630148999999999, 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1358 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.210461+0000", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.210461+0000", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.210460+0000", 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:55.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.216914+0000", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0064547780000000004 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.216946+0000", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1489999999999998e-05 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.216956+0000", 2024-09-19T21:34:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0411000000000001e-05 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.217056+0000", 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9753999999999994e-05 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.217337+0000", 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028072299999999998 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.217981+0000", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000644617 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.222855+0000", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00487383 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.225979+0000", 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0031236979999999998 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.226043+0000", 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4012999999999999e-05 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.226091+0000", 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8226000000000002e-05 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:1360 3.0 3:042cefee:::rbd_data.110682966ae8.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14836736~39936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.226874+0000", 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8165500640000003, 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012368195, 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1360 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.226874+0000", 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.226874+0000", 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.226873+0000", 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.232241+0000", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0053687550000000002 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.232259+0000", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.802e-05 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.232269+0000", 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.764e-06 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.232342+0000", 2024-09-19T21:34:55.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3324999999999993e-05 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.232562+0000", 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021906200000000001 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 2", 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.233145+0000", 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00058295999999999999 2024-09-19T21:34:55.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.238845+0000", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0057001769999999998 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.239146+0000", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00030150400000000002 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.239210+0000", 2024-09-19T21:34:55.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3786000000000002e-05 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.239242+0000", 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2187000000000002e-05 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2647 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467049+0000", 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5763749239999996, 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011115997000000001, 2024-09-19T21:34:55.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2647 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467049+0000", 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467049+0000", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467049+0000", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467052+0000", 2024-09-19T21:34:55.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5880000000000002e-06 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467061+0000", 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0469999999999999e-06 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467068+0000", 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.392e-06 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478055+0000", 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010987223000000001 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478104+0000", 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8872999999999998e-05 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478165+0000", 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1336999999999999e-05 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2648 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467073+0000", 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5763508929999999, 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011246803, 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2648 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.083 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467073+0000", 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467073+0000", 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467073+0000", 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467076+0000", 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.118e-06 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467083+0000", 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4120000000000004e-06 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467089+0000", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9009999999999999e-06 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478198+0000", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011108758999999999 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478256+0000", 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8542000000000003e-05 2024-09-19T21:34:55.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478320+0000", 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3514000000000002e-05 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2649 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467095+0000", 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5763292050000004, 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011367544, 2024-09-19T21:34:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2649 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467095+0000", 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467095+0000", 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467094+0000", 2024-09-19T21:34:55.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467107+0000", 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2867999999999999e-05 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467116+0000", 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4470000000000006e-06 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467122+0000", 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0599999999999996e-06 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478350+0000", 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011228311 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478399+0000", 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9332000000000001e-05 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478462+0000", 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2920000000000001e-05 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2650 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467128+0000", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5762960619999999, 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011487387, 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2650 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467128+0000", 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467128+0000", 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467127+0000", 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467131+0000", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7979999999999999e-06 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467139+0000", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7670000000000003e-06 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467145+0000", 2024-09-19T21:34:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3690000000000003e-06 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478495+0000", 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011349569 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478538+0000", 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3300000000000002e-05 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478615+0000", 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7195999999999997e-05 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2651 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 10551296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467151+0000", 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5762726929999999, 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011603034999999999, 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2651 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467151+0000", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467151+0000", 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467151+0000", 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467154+0000", 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1379999999999999e-06 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467161+0000", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1810000000000004e-06 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467167+0000", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4369999999999998e-06 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478646+0000", 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011479339 2024-09-19T21:34:55.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478692+0000", 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6437999999999998e-05 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478754+0000", 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1921999999999994e-05 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2652 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 10747904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467172+0000", 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5762517279999999, 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011739220999999999, 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2652 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.102 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467172+0000", 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467172+0000", 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467172+0000", 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467175+0000", 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.439e-06 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467182+0000", 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9589999999999998e-06 2024-09-19T21:34:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467188+0000", 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4779999999999999e-06 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478791+0000", 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011603474000000001 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478849+0000", 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8106000000000002e-05 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478912+0000", 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2199000000000005e-05 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2653 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 13041664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467193+0000", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5762307130000002, 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011871955, 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2653 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467193+0000", 2024-09-19T21:34:55.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467193+0000", 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467193+0000", 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467196+0000", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.658e-06 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467204+0000", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5800000000000003e-06 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467210+0000", 2024-09-19T21:34:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2090000000000002e-06 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478945+0000", 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011734593999999999 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.478990+0000", 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4966999999999998e-05 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479065+0000", 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5613999999999998e-05 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2654 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 13238272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467217+0000", 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5762072759999999, 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012005406999999999, 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2654 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467217+0000", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467217+0000", 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467216+0000", 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467220+0000", 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8589999999999998e-06 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467227+0000", 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5639999999999999e-06 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467236+0000", 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4130000000000006e-06 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479101+0000", 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011865224000000001 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479161+0000", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0216000000000001e-05 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479222+0000", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0952999999999997e-05 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2655 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467319+0000", 2024-09-19T21:34:55.115 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5761046180000005, 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012053094, 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2655 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467319+0000", 2024-09-19T21:34:55.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467319+0000", 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467319+0000", 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467325+0000", 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2639999999999997e-06 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467339+0000", 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3864e-05 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467347+0000", 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0809999999999994e-06 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479263+0000", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.011915667 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479306+0000", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3856999999999997e-05 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479373+0000", 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6141000000000003e-05 2024-09-19T21:34:55.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2657 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467354+0000", 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5760695790000003, 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012175177000000001, 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.120 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2657 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467354+0000", 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467354+0000", 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467354+0000", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467358+0000", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2599999999999999e-06 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467366+0000", 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0590000000000003e-06 2024-09-19T21:34:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467373+0000", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479402+0000", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012029285000000001 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479467+0000", 2024-09-19T21:34:55.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4688000000000001e-05 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479530+0000", 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2582999999999994e-05 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2658 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 15925248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467378+0000", 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5760460409999997, 2024-09-19T21:34:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.013181521, 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2658 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467378+0000", 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467378+0000", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467377+0000", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467380+0000", 2024-09-19T21:34:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467386+0000", 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6670000000000001e-06 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467389+0000", 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5659999999999998e-06 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479563+0000", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012173409 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479606+0000", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.303e-05 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480560+0000", 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00095375099999999997 2024-09-19T21:34:55.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2659 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467392+0000", 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5760317769999999, 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012442768, 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.129 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2659 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467392+0000", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467392+0000", 2024-09-19T21:34:55.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467392+0000", 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467394+0000", 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.131e-06 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467398+0000", 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7220000000000001e-06 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467401+0000", 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479708+0000", 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012306816999999999 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479770+0000", 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2265000000000001e-05 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479835+0000", 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4589000000000001e-05 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2660 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467404+0000", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5760198970000001, 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012579803000000001, 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2660 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467404+0000", 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467404+0000", 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467404+0000", 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467406+0000", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1959999999999998e-06 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467410+0000", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4009999999999999e-06 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467414+0000", 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3909999999999998e-06 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479863+0000", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012449205 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479914+0000", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1229e-05 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.479984+0000", 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9670000000000002e-05 2024-09-19T21:34:55.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2662 3.12 3:4bc45645:::rbd_data.1106e9705673.0000000000000003:9 [sparse-read 16515072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.467418+0000", 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5760063820000001, 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012732818, 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2662 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467418+0000", 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467418+0000", 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467417+0000", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467419+0000", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.052e-06 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467423+0000", 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6619999999999998e-06 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.467427+0000", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480014+0000", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.012587862 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480074+0000", 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9737000000000001e-05 2024-09-19T21:34:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480150+0000", 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6298999999999996e-05 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4720.0:2800 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:head [stat,truncate 15990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:50.468093+0000", 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5753305949999996, 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.014273971999999999, 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:55.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4720", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1958783801", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2800 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468093+0000", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468093+0000", 2024-09-19T21:34:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468092+0000", 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468100+0000", 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7149999999999998e-06 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:55.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468127+0000", 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7019999999999999e-05 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.468137+0000", 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0274e-05 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480181+0000", 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.01204416 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480228+0000", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6814999999999998e-05 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480318+0000", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9928999999999999e-05 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.480969+0000", 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00065117799999999998 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.482307+0000", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00133837 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.482346+0000", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8469e-05 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:50.482367+0000", 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1447e-05 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.147 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:55.148 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:55.148 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:55.149 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:55.145+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:55.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:55.250 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:55.245+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:55.274 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:55.274 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:55.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:55.351 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:55.345+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:55.393 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:55.393 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:55.393 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:55.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:55.452 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:55.449+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:55.509 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.246743603000001, 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:34:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:34:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.245417134, 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:34:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.24538566, 2024-09-19T21:34:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:34:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:34:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.245346705999999, 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:34:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.245305663, 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:34:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:34:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4822 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.236062119, 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021034206999999999, 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4822 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268248+0000", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268249+0000", 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.316e-06 2024-09-19T21:34:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268252+0000", 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268255+0000", 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9629999999999998e-06 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289149+0000", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.020894302 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289216+0000", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7269999999999998e-05 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289282+0000", 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5901999999999993e-05 2024-09-19T21:34:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.236053279, 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.236043049999999, 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:34:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.236033608, 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:34:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.236015168, 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:34:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.23600456, 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:34:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235995820999999, 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:34:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235983572, 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:34:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235784757999999, 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235766894999999, 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235750349, 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:34:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235734003999999, 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235717126999999, 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235698940000001, 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:34:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:34:55.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:55.598 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.235680411000001, 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:55.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:34:55.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:55.603 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:55.604 INFO:teuthology.orchestra.run.smithi122.stdout:1076 read 0xbbc00 thru 0xcae1f (0xf220 bytes) 2024-09-19T21:34:55.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:55.605 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:55.601+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:55.648 INFO:teuthology.orchestra.run.smithi122.stdout:1077 trunc from 0x2993600 to 0xa288a00 2024-09-19T21:34:55.684 INFO:teuthology.orchestra.run.smithi122.stdout:1078 punch from 0x526b000 to 0x5279800, (0xe800 bytes) 2024-09-19T21:34:55.690 INFO:teuthology.orchestra.run.smithi122.stdout:1079 punch from 0x1ac9000 to 0x1ad6600, (0xd600 bytes) 2024-09-19T21:34:55.699 INFO:teuthology.orchestra.run.smithi122.stdout:1080 punch from 0x723d400 to 0x724b800, (0xe400 bytes) 2024-09-19T21:34:55.699 INFO:teuthology.orchestra.run.smithi122.stdout:1081 punch from 0x3522800 to 0x3528400, (0x5c00 bytes) 2024-09-19T21:34:55.705 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:55.701+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:55.710 INFO:teuthology.orchestra.run.smithi122.stdout:1084 punch from 0x1f02000 to 0x1f04400, (0x2400 bytes) 2024-09-19T21:34:55.736 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.736 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:55.736 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:55.736 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:55.746 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:55.806 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:55.801+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:55.866 INFO:teuthology.orchestra.run.smithi122.stdout:1085 write 0xc331000 thru 0xc33d3c3 (0xc3c4 bytes) 2024-09-19T21:34:55.867 INFO:teuthology.orchestra.run.smithi122.stdout:1086 write 0x18af600 thru 0x18b5f2e (0x692f bytes) 2024-09-19T21:34:55.868 INFO:teuthology.orchestra.run.smithi122.stdout:1087 read 0x5e48200 thru 0x5e4c55a (0x435b bytes) 2024-09-19T21:34:55.869 INFO:teuthology.orchestra.run.smithi122.stdout:1089 write 0x4951400 thru 0x4953b98 (0x2799 bytes) 2024-09-19T21:34:55.870 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.870 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:55.870 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:55.870 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:55.870 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:55.871 INFO:teuthology.orchestra.run.smithi122.stdout:1090 read 0x7f94800 thru 0x7f9b4f8 (0x6cf9 bytes) 2024-09-19T21:34:55.871 INFO:teuthology.orchestra.run.smithi122.stdout:1093 punch from 0xb512600 to 0xb521600, (0xf000 bytes) 2024-09-19T21:34:55.881 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:55.887 INFO:teuthology.orchestra.run.smithi122.stdout:1094 trunc from 0xc33d400 to 0x6989600 2024-09-19T21:34:55.907 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:55.901+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:55.929 INFO:teuthology.orchestra.run.smithi122.stdout:1098 punch from 0x5cbea00 to 0x5cc9c00, (0xb200 bytes) 2024-09-19T21:34:55.932 INFO:teuthology.orchestra.run.smithi122.stdout:1099 read 0x1894600 thru 0x189c1bf (0x7bc0 bytes) 2024-09-19T21:34:55.934 INFO:teuthology.orchestra.run.smithi122.stdout:1101 punch from 0x6cb200 to 0x6ce600, (0x3400 bytes) 2024-09-19T21:34:55.934 INFO:teuthology.orchestra.run.smithi122.stdout:1103 write 0xa4d200 thru 0xa59000 (0xbe01 bytes) 2024-09-19T21:34:55.972 INFO:teuthology.orchestra.run.smithi122.stdout:1105 write 0x9eb8800 thru 0x9ec2779 (0x9f7a bytes) 2024-09-19T21:34:55.974 INFO:teuthology.orchestra.run.smithi122.stdout:1106 read 0x4b5f600 thru 0x4b5fcc5 (0x6c6 bytes) 2024-09-19T21:34:55.975 INFO:teuthology.orchestra.run.smithi122.stdout:1108 punch from 0x2c39e00 to 0x2c3c000, (0x2200 bytes) 2024-09-19T21:34:55.991 INFO:teuthology.orchestra.run.smithi122.stdout:1109 trunc from 0x9ec2800 to 0x3e5dc00 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:55.992 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.055235545000002, 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:55.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:55.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:34:55.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:55.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:55.999 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4788 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.719009816, 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053208699999999998, 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.000 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4788 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269024+0000", 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269027+0000", 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0889999999999998e-06 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269032+0000", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.31e-06 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269050+0000", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8138000000000002e-05 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322058+0000", 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053008161999999998 2024-09-19T21:34:56.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322126+0000", 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8269999999999995e-05 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322233+0000", 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107027 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4789 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 423936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718979244, 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053201143999999999, 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4789 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.005 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:34:56.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269066+0000", 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1392e-05 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269073+0000", 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0500000000000003e-06 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269079+0000", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322103+0000", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053024486000000003 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322168+0000", 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949000000000004e-05 2024-09-19T21:34:56.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322256+0000", 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8187000000000001e-05 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4790 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718951415999999, 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053307251999999999, 2024-09-19T21:34:56.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4790 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269097+0000", 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4481000000000001e-05 2024-09-19T21:34:56.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0243e-05 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0139999999999998e-06 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322255+0000", 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053142366000000003 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322323+0000", 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7725999999999995e-05 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322390+0000", 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7800999999999997e-05 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.013 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4791 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718917532000001, 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053280393000000002, 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4791 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269119+0000", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269125+0000", 2024-09-19T21:34:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5180000000000004e-06 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269129+0000", 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6990000000000002e-06 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322279+0000", 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053149890999999998 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322342+0000", 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2392999999999997e-05 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322397+0000", 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5798999999999999e-05 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4792 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.71890157, 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053414182999999997, 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4792 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269141+0000", 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6810000000000003e-06 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3549999999999998e-06 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322416+0000", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053270709999999999 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322474+0000", 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8161000000000001e-05 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322547+0000", 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3357999999999998e-05 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4796 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718885922, 2024-09-19T21:34:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053558802000000003, 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4796 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269148+0000", 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0439999999999998e-06 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269155+0000", 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.967e-06 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269167+0000", 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1935e-05 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322574+0000", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053407102999999997 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322642+0000", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7683999999999998e-05 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322707+0000", 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5421999999999998e-05 2024-09-19T21:34:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4799 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718862923, 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054302238000000003, 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4799 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269171+0000", 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0429999999999999e-06 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269186+0000", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9840000000000001e-06 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323330+0000", 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054144172999999997 2024-09-19T21:34:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323399+0000", 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9543999999999997e-05 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323474+0000", 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4564e-05 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4868 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718844367999999, 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053685682999999998, 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4868 2024-09-19T21:34:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269192+0000", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2639999999999998e-06 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269198+0000", 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8810000000000001e-06 2024-09-19T21:34:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1049999999999997e-06 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322728+0000", 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053525831000000003 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322792+0000", 2024-09-19T21:34:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4367000000000002e-05 2024-09-19T21:34:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322876+0000", 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3558000000000002e-05 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4869 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718828325, 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053842947000000002, 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4869 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269208+0000", 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3879999999999998e-06 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269214+0000", 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269221+0000", 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.657e-06 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322897+0000", 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053676662 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322966+0000", 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9085999999999996e-05 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323049+0000", 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2742000000000002e-05 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4870 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.718624553, 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053808917999999997, 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4870 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269225+0000", 2024-09-19T21:34:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998155 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269413+0000", 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018773800000000001 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269420+0000", 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7399999999999998e-06 2024-09-19T21:34:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269424+0000", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9389999999999999e-06 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323071+0000", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053647087000000003 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323137+0000", 2024-09-19T21:34:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5810999999999997e-05 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323219+0000", 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2236000000000003e-05 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4871 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717983577, 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053346843999999997, 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4871 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269436+0000", 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9993844 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270057+0000", 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062110799999999999 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270066+0000", 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2709999999999998e-06 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270073+0000", 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9770000000000003e-06 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323240+0000", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053167430000000002 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323312+0000", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1581000000000005e-05 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323398+0000", 2024-09-19T21:34:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5939999999999994e-05 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4872 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717955034999999, 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053465749, 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4872 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270079+0000", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270082+0000", 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9730000000000002e-06 2024-09-19T21:34:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270088+0000", 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270093+0000", 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323425+0000", 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053332349000000001 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323493+0000", 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7828000000000002e-05 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323545+0000", 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2166000000000002e-05 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4873 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 106496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717937222, 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053689477999999999, 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4873 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270099+0000", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270106+0000", 2024-09-19T21:34:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270111+0000", 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7520000000000004e-06 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323583+0000", 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053471779999999997 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323664+0000", 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1402000000000004e-05 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323787+0000", 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122404 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4874 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 116736~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717910147, 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053900483999999999, 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4874 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270131+0000", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9079999999999998e-06 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270138+0000", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5699999999999998e-06 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270143+0000", 2024-09-19T21:34:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323826+0000", 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053683467999999998 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323902+0000", 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5350000000000002e-05 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324025+0000", 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012334600000000001 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4875 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717874384, 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054010483999999997, 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4875 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270177+0000", 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6985000000000001e-05 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270188+0000", 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0993e-05 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270193+0000", 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7679999999999997e-06 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324068+0000", 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053874248 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324119+0000", 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1615e-05 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324171+0000", 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440999999999998e-05 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4876 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717589995999999, 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053834246000000002, 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4876 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270198+0000", 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997535 2024-09-19T21:34:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270450+0000", 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025216000000000003 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270460+0000", 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8409999999999995e-06 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270467+0000", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6680000000000004e-06 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324197+0000", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053730214999999998 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324239+0000", 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1755999999999998e-05 2024-09-19T21:34:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324279+0000", 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0224e-05 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4877 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717560718, 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054028259000000002, 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4877 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270473+0000", 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270476+0000", 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8590000000000001e-06 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270490+0000", 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4071e-05 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270496+0000", 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7599999999999999e-06 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324308+0000", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053811953000000003 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324349+0000", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0747999999999997e-05 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324502+0000", 2024-09-19T21:34:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015331599999999999 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4878 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.717513370000001, 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054078566000000002, 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4878 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270524+0000", 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271662+0000", 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001138114 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271683+0000", 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0638000000000001e-05 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324520+0000", 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052837566000000002 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324559+0000", 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8782999999999998e-05 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324600+0000", 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.066e-05 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4881 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.715986087999999, 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053152144999999998, 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4881 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272051+0000", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8890000000000002e-06 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272057+0000", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6030000000000004e-06 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272066+0000", 2024-09-19T21:34:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5540000000000003e-06 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325024+0000", 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052957551999999998 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325114+0000", 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0708e-05 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325201+0000", 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6236999999999994e-05 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.092 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:56.094 INFO:teuthology.orchestra.run.smithi122.stdout:1110 write 0x823ba00 thru 0x824aeaa (0xf4ab bytes) 2024-09-19T21:34:56.094 INFO:teuthology.orchestra.run.smithi122.stdout:1111 write 0x8c65200 thru 0x8c7411f (0xef20 bytes) 2024-09-19T21:34:56.094 INFO:teuthology.orchestra.run.smithi122.stdout:1112 flatten 2024-09-19T21:34:56.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:56.095 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:56.089+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:56.113 INFO:teuthology.orchestra.run.smithi122.stdout:1113 read 0x2612800 thru 0x261e800 (0xc001 bytes) 2024-09-19T21:34:56.121 INFO:teuthology.orchestra.run.smithi122.stdout:1116 trunc from 0x8c74200 to 0x75e5a00 2024-09-19T21:34:56.155 INFO:teuthology.orchestra.run.smithi122.stdout:1117 read 0x63e1400 thru 0x63e8c56 (0x7857 bytes) 2024-09-19T21:34:56.156 INFO:teuthology.orchestra.run.smithi122.stdout:1120 write 0x11bd400 thru 0x11c665c (0x925d bytes) 2024-09-19T21:34:56.158 INFO:teuthology.orchestra.run.smithi122.stdout:1121 read 0x48c9200 thru 0x48d4b30 (0xb931 bytes) 2024-09-19T21:34:56.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:34:56.159 INFO:teuthology.orchestra.run.smithi122.stdout:1122 read 0xd85600 thru 0xd8994c (0x434d bytes) 2024-09-19T21:34:56.162 INFO:teuthology.orchestra.run.smithi122.stdout:1123 punch from 0x54aea00 to 0x54b5000, (0x6600 bytes) 2024-09-19T21:34:56.163 INFO:teuthology.orchestra.run.smithi122.stdout:1126 punch from 0x54ca000 to 0x54d3200, (0x9200 bytes) 2024-09-19T21:34:56.168 INFO:teuthology.orchestra.run.smithi122.stdout:1127 trunc from 0x75e5a00 to 0x8ed6c00 2024-09-19T21:34:56.187 INFO:teuthology.orchestra.run.smithi122.stdout:1128 read 0x43cd200 thru 0x43d7b04 (0xa905 bytes) 2024-09-19T21:34:56.190 INFO:teuthology.orchestra.run.smithi122.stdout:1133 write 0x8244600 thru 0x8245dd7 (0x17d8 bytes) 2024-09-19T21:34:56.190 INFO:teuthology.orchestra.run.smithi122.stdout:1134 punch from 0x8d78800 to 0x8d7e200, (0x5a00 bytes) 2024-09-19T21:34:56.190 INFO:teuthology.orchestra.run.smithi122.stdout:1135 clone 9 order 21 su 65536 sc 2 2024-09-19T21:34:56.205 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:56.201+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:56.217 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:56.217 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:34:56.218 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:56.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:56.296 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:56.289+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:56.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:34:56.338 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:56.338 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:56.338 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:56.339 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:56.339 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:56.349 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:56.397 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:56.393+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:56.465 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:56.486 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:56.498 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:56.493+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.502 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.270127702, 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:56.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:34:56.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1446 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79635786799999997, 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028575401, 2024-09-19T21:34:56.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1446 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701217+0000", 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4450000000000001e-06 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701232+0000", 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5211999999999999e-05 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701236+0000", 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8689999999999997e-06 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729646+0000", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028410086000000001 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729707+0000", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0690999999999999e-05 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729786+0000", 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9327999999999997e-05 2024-09-19T21:34:56.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1456 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79622700700000004, 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028614296000000001, 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1456 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701341+0000", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701345+0000", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6969999999999999e-06 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701351+0000", 2024-09-19T21:34:56.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.122e-06 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701368+0000", 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7487999999999999e-05 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729825+0000", 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028456535000000002 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729885+0000", 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9738999999999998e-05 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729956+0000", 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1496999999999997e-05 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1457 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.516 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79619668300000002, 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028756075999999998, 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1457 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701374+0000", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9319999999999999e-06 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701377+0000", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.659e-06 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701380+0000", 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8650000000000001e-06 2024-09-19T21:34:56.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729994+0000", 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028613791 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730052+0000", 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8508999999999998e-05 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730128+0000", 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5683e-05 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1460 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79618651200000001, 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028920154999999999, 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1460 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701383+0000", 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2389999999999999e-06 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701386+0000", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.717e-06 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701389+0000", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8990000000000001e-06 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730165+0000", 2024-09-19T21:34:56.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02877569 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730227+0000", 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2791000000000002e-05 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730302+0000", 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5036999999999993e-05 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1461 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79617440100000003, 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02910503, 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1461 2024-09-19T21:34:56.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701397+0000", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.576e-06 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701402+0000", 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4160000000000003e-06 2024-09-19T21:34:56.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701407+0000", 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730338+0000", 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028931354999999999 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730404+0000", 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5630999999999996e-05 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730499+0000", 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5397999999999997e-05 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.530 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79609989400000003, 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:34:56.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79607006700000005, 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:34:56.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79606038099999998, 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:34:56.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:34:56.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:34:56.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79605205499999998, 2024-09-19T21:34:56.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:34:56.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79604166300000001, 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:56.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:34:56.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79603116699999998, 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.554 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:34:56.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1530 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.558 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79377519299999999, 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028447986000000001, 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1530 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703793+0000", 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703807+0000", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4080000000000001e-05 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703816+0000", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9670000000000007e-06 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703822+0000", 2024-09-19T21:34:56.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732054+0000", 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028232126999999999 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732131+0000", 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7206000000000006e-05 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732242+0000", 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110623 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1531 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79374199400000001, 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028589459000000001, 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1531 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703826+0000", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703835+0000", 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9910000000000008e-06 2024-09-19T21:34:56.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703844+0000", 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9390000000000003e-06 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703849+0000", 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.147e-06 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732283+0000", 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028433429999999999 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732339+0000", 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5804999999999998e-05 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732416+0000", 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7625000000000003e-05 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.567 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1534 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79371438699999997, 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028735752, 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1534 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703856+0000", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4669999999999998e-06 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703866+0000", 2024-09-19T21:34:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2909999999999996e-06 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703879+0000", 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3698e-05 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732452+0000", 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028572441000000001 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732514+0000", 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1785999999999994e-05 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732590+0000", 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6440000000000007e-05 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1541 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79368483499999998, 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028868288999999998, 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1541 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703890+0000", 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8199999999999999e-06 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703897+0000", 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703902+0000", 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1170000000000003e-06 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732626+0000", 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028723790999999999 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732683+0000", 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6645999999999999e-05 2024-09-19T21:34:56.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732752+0000", 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9426999999999998e-05 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1544 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79363784500000001, 2024-09-19T21:34:56.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028982658000000001, 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1544 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703930+0000", 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703933+0000", 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7539999999999998e-06 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703939+0000", 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703945+0000", 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0159999999999999e-06 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732784+0000", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028839773999999999 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732841+0000", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6759000000000003e-05 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732914+0000", 2024-09-19T21:34:56.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2564999999999994e-05 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1545 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79361638700000003, 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029130377999999998, 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1545 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703955+0000", 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6860000000000002e-06 2024-09-19T21:34:56.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703968+0000", 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3226e-05 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703974+0000", 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9510000000000003e-06 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732951+0000", 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028976842999999999 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733005+0000", 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4842000000000001e-05 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733083+0000", 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7380999999999997e-05 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.79358707900000003, 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:34:56.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:56.591 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.76836969300000002, 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:56.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:34:56.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:34:56.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:56.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:56.598 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:56.598 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:56.599 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:56.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:56.600 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:56.602 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:56.602 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:56.593+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:56.701 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:56.697+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:56.722 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:56.722 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:56.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:56.801 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:56.797+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:56.846 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:56.846 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:56.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:56.903 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:56.897+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:57.003 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:56.997+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:57.011 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.011 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.016 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-19T21:34:57.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:57.105 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:57.101+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:57.159 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.159 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:57.205 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:57.201+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:57.269 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.269 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.276 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:34:57.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:57.298 INFO:teuthology.orchestra.run.smithi122.stdout:1136 trunc from 0x8ed6c00 to 0xd3c7800 2024-09-19T21:34:57.332 INFO:teuthology.orchestra.run.smithi122.stdout:1140 read 0xca37e00 thru 0xca3c3ad (0x45ae bytes) 2024-09-19T21:34:57.334 INFO:teuthology.orchestra.run.smithi122.stdout:1146 write 0x78ec00 thru 0x79c705 (0xdb06 bytes) 2024-09-19T21:34:57.336 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:57.333+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:57.338 INFO:teuthology.orchestra.run.smithi122.stdout:1149 write 0x21f2a00 thru 0x21fd673 (0xac74 bytes) 2024-09-19T21:34:57.339 INFO:teuthology.orchestra.run.smithi122.stdout:1150 punch from 0x7cfc600 to 0x7d02200, (0x5c00 bytes) 2024-09-19T21:34:57.342 INFO:teuthology.orchestra.run.smithi122.stdout:1151 read 0xb4f0800 thru 0xb4f6100 (0x5901 bytes) 2024-09-19T21:34:57.343 INFO:teuthology.orchestra.run.smithi122.stdout:1155 read 0x5f6d200 thru 0x5f7b65d (0xe45e bytes) 2024-09-19T21:34:57.366 INFO:teuthology.orchestra.run.smithi122.stdout:1156 read 0xb607400 thru 0xb608dea (0x19eb bytes) 2024-09-19T21:34:57.372 INFO:teuthology.orchestra.run.smithi122.stdout:1161 trunc from 0xd3c7800 to 0xaf2f000 2024-09-19T21:34:57.387 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.387 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:34:57.407 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:57.401+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:57.407 INFO:teuthology.orchestra.run.smithi122.stdout:1162 read 0x467c400 thru 0x4682067 (0x5c68 bytes) 2024-09-19T21:34:57.447 INFO:teuthology.orchestra.run.smithi122.stdout:1163 write 0xec8f000 thru 0xec9a733 (0xb734 bytes) 2024-09-19T21:34:57.455 INFO:teuthology.orchestra.run.smithi122.stdout:1164 trunc from 0xec9a800 to 0xc968000 2024-09-19T21:34:57.495 INFO:teuthology.orchestra.run.smithi122.stdout:1166 punch from 0xbc3a400 to 0xbc46c00, (0xc800 bytes) 2024-09-19T21:34:57.497 INFO:teuthology.orchestra.run.smithi122.stdout:1168 read 0xb1cd200 thru 0xb1cf145 (0x1f46 bytes) 2024-09-19T21:34:57.501 INFO:teuthology.orchestra.run.smithi122.stdout:1169 trunc from 0xc968000 to 0xe3d4400 2024-09-19T21:34:57.508 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:57.501+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:57.508 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.509 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:34:57.525 INFO:teuthology.orchestra.run.smithi122.stdout:1170 punch from 0x6e4b000 to 0x6e4e200, (0x3200 bytes) 2024-09-19T21:34:57.526 INFO:teuthology.orchestra.run.smithi122.stdout:1171 write 0x554e000 thru 0x5555098 (0x7099 bytes) 2024-09-19T21:34:57.527 INFO:teuthology.orchestra.run.smithi122.stdout:1172 read 0x3581e00 thru 0x358d589 (0xb78a bytes) 2024-09-19T21:34:57.530 INFO:teuthology.orchestra.run.smithi122.stdout:1173 write 0x3e25c00 thru 0x3e3044d (0xa84e bytes) 2024-09-19T21:34:57.533 INFO:teuthology.orchestra.run.smithi122.stdout:1175 write 0xda07400 thru 0xda127dc (0xb3dd bytes) 2024-09-19T21:34:57.533 INFO:teuthology.orchestra.run.smithi122.stdout:1176 punch from 0xcea200 to 0xcec600, (0x2400 bytes) 2024-09-19T21:34:57.536 INFO:teuthology.orchestra.run.smithi122.stdout:1177 read 0x81e1800 thru 0x81eb694 (0x9e95 bytes) 2024-09-19T21:34:57.600 INFO:teuthology.orchestra.run.smithi122.stdout:1178 trunc from 0xe3d4400 to 0xc17e00 2024-09-19T21:34:57.610 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:57.605+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:57.653 INFO:teuthology.orchestra.run.smithi122.stdout:1180 punch from 0x872000 to 0x878a00, (0x6a00 bytes) 2024-09-19T21:34:57.675 INFO:teuthology.orchestra.run.smithi122.stdout:1181 trunc from 0xc17e00 to 0x95d7e00 2024-09-19T21:34:57.681 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.681 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:34:57.699 INFO:teuthology.orchestra.run.smithi122.stdout:1182 punch from 0x7432c00 to 0x7433c00, (0x1000 bytes) 2024-09-19T21:34:57.700 INFO:teuthology.orchestra.run.smithi122.stdout:1187 read 0x42a0000 thru 0x42a3a7a (0x3a7b bytes) 2024-09-19T21:34:57.710 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:57.705+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:57.723 INFO:teuthology.orchestra.run.smithi122.stdout:1189 trunc from 0x95d7e00 to 0x740a00 2024-09-19T21:34:57.761 INFO:teuthology.orchestra.run.smithi122.stdout:1191 read 0xd3400 thru 0xdca7f (0x9680 bytes) 2024-09-19T21:34:57.767 INFO:teuthology.orchestra.run.smithi122.stdout:1192 punch from 0x23bc00 to 0x245e00, (0xa200 bytes) 2024-09-19T21:34:57.768 INFO:teuthology.orchestra.run.smithi122.stdout:1193 punch from 0x1b2400 to 0x1bdc00, (0xb800 bytes) 2024-09-19T21:34:57.768 INFO:teuthology.orchestra.run.smithi122.stdout:1194 punch from 0x126200 to 0x130c00, (0xaa00 bytes) 2024-09-19T21:34:57.800 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.800 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.811 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:34:57.812 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:57.805+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:57.821 INFO:teuthology.orchestra.run.smithi122.stdout:1195 write 0x477c200 thru 0x47804fc (0x42fd bytes) 2024-09-19T21:34:57.823 INFO:teuthology.orchestra.run.smithi122.stdout:1196 read 0x3c2a800 thru 0x3c35871 (0xb072 bytes) 2024-09-19T21:34:57.824 INFO:teuthology.orchestra.run.smithi122.stdout:1197 read 0x2e3f000 thru 0x2e45935 (0x6936 bytes) 2024-09-19T21:34:57.824 INFO:teuthology.orchestra.run.smithi122.stdout:1198 read 0x2258e00 thru 0x226765a (0xe85b bytes) 2024-09-19T21:34:57.824 INFO:teuthology.orchestra.run.smithi122.stdout:1199 read 0x1577400 thru 0x1578573 (0x1174 bytes) 2024-09-19T21:34:57.824 INFO:teuthology.orchestra.run.smithi122.stdout:1200 read 0x99fa00 thru 0x9aeaa1 (0xf0a2 bytes) 2024-09-19T21:34:57.830 INFO:teuthology.orchestra.run.smithi122.stdout:1203 trunc from 0x4780600 to 0x66b2200 2024-09-19T21:34:57.862 INFO:teuthology.orchestra.run.smithi122.stdout:1207 read 0x301f600 thru 0x3020cc4 (0x16c5 bytes) 2024-09-19T21:34:57.874 INFO:teuthology.orchestra.run.smithi122.stdout:1208 trunc from 0x66b2200 to 0x2726a00 2024-09-19T21:34:57.911 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:57.905+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:57.939 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:57.939 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:57.941 INFO:teuthology.orchestra.run.smithi122.stdout:1209 write 0x882d200 thru 0x883b5c3 (0xe3c4 bytes) 2024-09-19T21:34:57.942 INFO:teuthology.orchestra.run.smithi122.stdout:1210 clone 10 order 21 su 131072 sc 3 2024-09-19T21:34:57.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:34:58.012 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:58.009+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:58.054 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.054 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.064 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:34:58.113 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:58.109+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:58.168 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.169 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:34:58.214 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:58.209+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:58.292 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.292 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:34:58.315 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:58.309+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:58.409 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.410 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.416 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:58.409+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:58.425 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:34:58.431 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone9 from 0x883b600 (overlap 0x740a00) to 0x0 2024-09-19T21:34:58.517 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:34:58.513+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:34:58.579 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.579 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:34:58.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:34:58.618 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:58.613+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:58.696 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:34:58.696 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:34:58.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:34:58.738 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:34:58.739 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:58.733+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:58.766 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:34:58.785 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:34:58.815 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:34:58.820 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:58.817+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:58.839 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:34:58.847 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:58.847 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:34:58.847 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:34:58.847 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:34:58.847 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:58.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:34:58.868 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:34:58.870 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:34:58.924 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:58.921+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:58.977 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:34:58.977 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:34:58.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 17.745031647000001, 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:34:58.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:58.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:34:58.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1446 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2712618130000002, 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028575401, 2024-09-19T21:34:58.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1446 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:58.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701217+0000", 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4450000000000001e-06 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701232+0000", 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5211999999999999e-05 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701236+0000", 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8689999999999997e-06 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729646+0000", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028410086000000001 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729707+0000", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0690999999999999e-05 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729786+0000", 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9327999999999997e-05 2024-09-19T21:34:58.986 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1456 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.271130952, 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028614296000000001, 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:58.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1456 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701341+0000", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701345+0000", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6969999999999999e-06 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701351+0000", 2024-09-19T21:34:58.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.122e-06 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701368+0000", 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7487999999999999e-05 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729825+0000", 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028456535000000002 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729885+0000", 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9738999999999998e-05 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729956+0000", 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1496999999999997e-05 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.991 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1457 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2711006280000001, 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028756075999999998, 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1457 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:58.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:34:58.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701374+0000", 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9319999999999999e-06 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701377+0000", 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.659e-06 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701380+0000", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8650000000000001e-06 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729994+0000", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028613791 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730052+0000", 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8508999999999998e-05 2024-09-19T21:34:58.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730128+0000", 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5683e-05 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1460 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2710904570000001, 2024-09-19T21:34:58.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028920154999999999, 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1460 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:58.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701383+0000", 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2389999999999999e-06 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701386+0000", 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.717e-06 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701389+0000", 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8990000000000001e-06 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:58.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730165+0000", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02877569 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730227+0000", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2791000000000002e-05 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730302+0000", 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5036999999999993e-05 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.000 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1461 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2710783459999999, 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02910503, 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1461 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701397+0000", 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.576e-06 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701402+0000", 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4160000000000003e-06 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701407+0000", 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730338+0000", 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028931354999999999 2024-09-19T21:34:59.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730404+0000", 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5630999999999996e-05 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730499+0000", 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5397999999999997e-05 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.271003839, 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.006 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:34:59.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:59.010 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2709740119999999, 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:59.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:34:59.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2709643260000001, 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.015 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:34:59.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:34:59.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.270956, 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.020 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:34:59.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2709456079999999, 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:34:59.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.029 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2709351120000001, 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:34:59.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:34:59.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.034 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1530 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.268679138, 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028447986000000001, 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1530 2024-09-19T21:34:59.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703793+0000", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703807+0000", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4080000000000001e-05 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703816+0000", 2024-09-19T21:34:59.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9670000000000007e-06 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703822+0000", 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732054+0000", 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028232126999999999 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732131+0000", 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7206000000000006e-05 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732242+0000", 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110623 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.039 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1531 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2686459389999998, 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028589459000000001, 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1531 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703826+0000", 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703835+0000", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9910000000000008e-06 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703844+0000", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9390000000000003e-06 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703849+0000", 2024-09-19T21:34:59.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.147e-06 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732283+0000", 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028433429999999999 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732339+0000", 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5804999999999998e-05 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732416+0000", 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7625000000000003e-05 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1534 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.268618332, 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028735752, 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.044 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1534 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:59.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703856+0000", 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4669999999999998e-06 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703866+0000", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2909999999999996e-06 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703879+0000", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3698e-05 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732452+0000", 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028572441000000001 2024-09-19T21:34:59.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732514+0000", 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1785999999999994e-05 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732590+0000", 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6440000000000007e-05 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1541 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.26858878, 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028868288999999998, 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1541 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.049 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703890+0000", 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8199999999999999e-06 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703897+0000", 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703902+0000", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1170000000000003e-06 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732626+0000", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028723790999999999 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732683+0000", 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6645999999999999e-05 2024-09-19T21:34:59.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732752+0000", 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9426999999999998e-05 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1544 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.26854179, 2024-09-19T21:34:59.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028982658000000001, 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1544 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703930+0000", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703933+0000", 2024-09-19T21:34:59.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7539999999999998e-06 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703939+0000", 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703945+0000", 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0159999999999999e-06 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732784+0000", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028839773999999999 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732841+0000", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6759000000000003e-05 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732914+0000", 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2564999999999994e-05 2024-09-19T21:34:59.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1545 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.268520332, 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029130377999999998, 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1545 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703955+0000", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6860000000000002e-06 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703968+0000", 2024-09-19T21:34:59.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3226e-05 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703974+0000", 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9510000000000003e-06 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732951+0000", 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028976842999999999 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733005+0000", 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4842000000000001e-05 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733083+0000", 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7380999999999997e-05 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.062 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2684910240000002, 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:59.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:34:59.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:34:59.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.2432736379999998, 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:34:59.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:34:59.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:34:59.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:34:59.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:34:59.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:34:59.073 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:34:59.074 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:34:59.074 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:34:59.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:34:59.075 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-19T21:34:59.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:34:59.076 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:59.069+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:59.175 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:34:59.169+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:34:59.210 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.210 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:59.211 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:59.211 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:59.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:34:59.276 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:59.269+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:59.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:34:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.335 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:59.335 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:59.335 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:59.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:34:59.392 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:59.389+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:34:59.442 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:34:59.453 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.453 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4296 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196494344000001, 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022203509999999999, 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4296 2024-09-19T21:34:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253833+0000", 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.213e-06 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253840+0000", 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5930000000000002e-06 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253845+0000", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3979999999999998e-06 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275901+0000", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022056172999999998 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275964+0000", 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2723000000000002e-05 2024-09-19T21:34:59.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276034+0000", 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9774e-05 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4297 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196474483999999, 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359332999999999, 2024-09-19T21:34:59.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4297 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253852+0000", 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4779999999999998e-06 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253858+0000", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7740000000000001e-06 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253860+0000", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276057+0000", 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022196344999999999 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276120+0000", 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3521000000000004e-05 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276209+0000", 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9135999999999995e-05 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4298 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196459934, 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022554268999999998, 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4298 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253864+0000", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253873+0000", 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0289999999999994e-06 2024-09-19T21:34:59.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253880+0000", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4729999999999997e-06 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253888+0000", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276235+0000", 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022346462000000001 2024-09-19T21:34:59.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276323+0000", 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7990000000000003e-05 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276419+0000", 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6291000000000004e-05 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4299 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 335872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196422336000001, 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022708860000000001, 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4299 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253905+0000", 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.591e-06 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253921+0000", 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5282e-05 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253927+0000", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.003e-06 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276468+0000", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022541017 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276535+0000", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7119000000000006e-05 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276611+0000", 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6347000000000006e-05 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4300 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196394652999999, 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022876461000000001, 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4300 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253932+0000", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.857e-06 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253935+0000", 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:34:59.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253938+0000", 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.441e-06 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276636+0000", 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022698381 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276702+0000", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5258000000000004e-05 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276806+0000", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010488399999999999 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4301 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:59.475 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196384653999999, 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023114603000000001, 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4301 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:59.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253941+0000", 2024-09-19T21:34:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.44e-06 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253944+0000", 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0960000000000001e-06 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253947+0000", 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4109999999999999e-06 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276839+0000", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022892550000000001 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276947+0000", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107793 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277055+0000", 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010752299999999999 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4302 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196150692, 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023063824, 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4302 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.480 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254173+0000", 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254179+0000", 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0789999999999997e-06 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254190+0000", 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0532e-05 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254196+0000", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6560000000000003e-06 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277079+0000", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022882731 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277146+0000", 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6934000000000007e-05 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277238+0000", 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1891999999999994e-05 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4303 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196122118000002, 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023222402, 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4303 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254205+0000", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8329999999999998e-06 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254212+0000", 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6549999999999998e-06 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254217+0000", 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277263+0000", 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023045955 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277333+0000", 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0257000000000002e-05 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277425+0000", 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1952000000000003e-05 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.488 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4304 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196079033, 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023378263, 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4304 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254245+0000", 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254248+0000", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2940000000000001e-06 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254256+0000", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254261+0000", 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5310000000000001e-06 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277449+0000", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023187433 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277538+0000", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9474000000000004e-05 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277624+0000", 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5560000000000001e-05 2024-09-19T21:34:59.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4305 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 446464~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.196054214, 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033521316000000002, 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4305 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254273+0000", 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0850000000000001e-06 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254280+0000", 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0720000000000002e-06 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254285+0000", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3839999999999996e-06 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277648+0000", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0233627 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277727+0000", 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9128999999999994e-05 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.287792+0000", 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010064559000000001 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4307 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.195873683999999, 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027454658999999999, 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4307 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254450+0000", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254455+0000", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4560000000000002e-06 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254464+0000", 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8570000000000007e-06 2024-09-19T21:34:59.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254471+0000", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281760+0000", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027288818999999999 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281834+0000", 2024-09-19T21:34:59.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4019999999999999e-05 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281906+0000", 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1353999999999995e-05 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4309 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.195294834999999, 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027050399999999999, 2024-09-19T21:34:59.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4309 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255029+0000", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255035+0000", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6500000000000001e-06 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255046+0000", 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1382999999999999e-05 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255061+0000", 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.554e-05 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281929+0000", 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02686796 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282019+0000", 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9860999999999999e-05 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282080+0000", 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0891999999999999e-05 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.505 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4311 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.195112697999999, 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027091388000000001, 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4311 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255211+0000", 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255216+0000", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3319999999999999e-06 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255228+0000", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1202999999999999e-05 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255235+0000", 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0350000000000004e-06 2024-09-19T21:34:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282101+0000", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026866476 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282202+0000", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282303+0000", 2024-09-19T21:34:59.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100883 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4313 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.195073723, 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027229189000000001, 2024-09-19T21:34:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4313 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255250+0000", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255254+0000", 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0629999999999999e-06 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255262+0000", 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-06 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255269+0000", 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4050000000000003e-06 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282326+0000", 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027057184000000001 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282403+0000", 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6720999999999999e-05 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282480+0000", 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7001000000000004e-05 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.514 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4744 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181235034, 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022372296999999999, 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4744 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269089+0000", 2024-09-19T21:34:59.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269092+0000", 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269102+0000", 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0077e-05 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0289999999999996e-06 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291277+0000", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022170091999999999 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291356+0000", 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9241999999999998e-05 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291462+0000", 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105609 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4746 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181213330999999, 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022538171999999999, 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4746 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:59.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269123+0000", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0820000000000005e-06 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269128+0000", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0549999999999999e-06 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291488+0000", 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359917 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291558+0000", 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0339000000000005e-05 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291649+0000", 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1558000000000007e-05 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4747 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181191684000002, 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022815518999999999, 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4747 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0185e-05 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:34:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7169999999999999e-06 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291712+0000", 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022561748 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291806+0000", 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3826999999999995e-05 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291948+0000", 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014276600000000001 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4749 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181170436999999, 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023058227000000001, 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4749 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269156+0000", 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2230000000000001e-06 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269165+0000", 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8470000000000002e-06 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269170+0000", 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8489999999999997e-06 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291977+0000", 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022806918999999998 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1191000000000003e-05 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292212+0000", 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016450899999999999 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4751 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181150561999999, 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023246788000000001, 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4751 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6450000000000005e-06 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269191+0000", 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9550000000000004e-06 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269196+0000", 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8500000000000002e-06 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292239+0000", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023042450999999999 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292360+0000", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121804 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292421+0000", 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0467000000000002e-05 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4753 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.181122706, 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023417911999999999, 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4753 2024-09-19T21:34:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269204+0000", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.481e-06 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269213+0000", 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000005e-06 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269219+0000", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1919999999999996e-06 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292445+0000", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023226009999999998 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292517+0000", 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1898999999999997e-05 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292620+0000", 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103198 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.543 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:59.545 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-19T21:34:58.808095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622610264,"num_objects":468,"num_object_clones":144,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":468,"num_whiteouts":177,"num_read":49974,"num_read_kb":269700,"num_write":3158,"num_write_kb":61802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3056,"ondisk_log_size":3056,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":274476,"kb_used_data":79476,"kb_used_omap":9,"kb_used_meta":194806,"kb_avail":565956564,"statfs":{"total":579820584960,"available":579539521536,"internally_reserved":0,"allocated":81383424,"data_stored":79273623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":199482022},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11],"upper_bound":2},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":91217484,"num_objects":80,"num_object_clones":8,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":11915,"num_read_kb":152724,"num_write":369,"num_write_kb":29088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672860"},"pg_stats":[{"pgid":"3.1f","version":"44'186","reported_seq":1519,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.946623+0000","last_change":"2024-09-19T21:34:32.012011+0000","last_active":"2024-09-19T21:34:57.946623+0000","last_peered":"2024-09-19T21:34:57.946623+0000","last_clean":"2024-09-19T21:34:57.946623+0000","last_became_active":"2024-09-19T21:34:30.013688+0000","last_became_peered":"2024-09-19T21:34:30.013688+0000","last_unstale":"2024-09-19T21:34:57.946623+0000","last_undegraded":"2024-09-19T21:34:57.946623+0000","last_fullsized":"2024-09-19T21:34:57.946623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:27:13.050028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":33867844,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":2897,"num_read_kb":4966,"num_write":174,"num_write_kb":16919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"42'56","reported_seq":434,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T21:34:54.353267+0000","last_change":"2024-09-19T21:34:32.006487+0000","last_active":"2024-09-19T21:34:54.353267+0000","last_peered":"2024-09-19T21:34:54.353267+0000","last_clean":"2024-09-19T21:34:54.353267+0000","last_became_active":"2024-09-19T21:34:30.011721+0000","last_became_peered":"2024-09-19T21:34:30.011721+0000","last_unstale":"2024-09-19T21:34:54.353267+0000","last_undegraded":"2024-09-19T21:34:54.353267+0000","last_fullsized":"2024-09-19T21:34:54.353267+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:21.923196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":10819584,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":8,"num_read":372,"num_read_kb":1454,"num_write":39,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"44'113","reported_seq":1797,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.556281+0000","last_change":"2024-09-19T21:34:32.024269+0000","last_active":"2024-09-19T21:34:57.556281+0000","last_peered":"2024-09-19T21:34:57.556281+0000","last_clean":"2024-09-19T21:34:57.556281+0000","last_became_active":"2024-09-19T21:34:30.010774+0000","last_became_peered":"2024-09-19T21:34:30.010774+0000","last_unstale":"2024-09-19T21:34:57.556281+0000","last_undegraded":"2024-09-19T21:34:57.556281+0000","last_fullsized":"2024-09-19T21:34:57.556281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:59.647828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":57513572,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":1924,"num_read_kb":9416,"num_write":147,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"44'213","reported_seq":3101,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.668235+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:34:57.668235+0000","last_peered":"2024-09-19T21:34:57.668235+0000","last_clean":"2024-09-19T21:34:57.668235+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:34:57.668235+0000","last_undegraded":"2024-09-19T21:34:57.668235+0000","last_fullsized":"2024-09-19T21:34:57.668235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":5195300,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":4078,"num_read_kb":4260,"num_write":159,"num_write_kb":350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"42'110","reported_seq":1773,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T21:34:55.731238+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:34:55.731238+0000","last_peered":"2024-09-19T21:34:55.731238+0000","last_clean":"2024-09-19T21:34:55.731238+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:34:55.731238+0000","last_undegraded":"2024-09-19T21:34:55.731238+0000","last_fullsized":"2024-09-19T21:34:55.731238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":59427475,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1753,"num_read_kb":14118,"num_write":108,"num_write_kb":2662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"42'68","reported_seq":1687,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.938597+0000","last_change":"2024-09-19T21:34:32.001087+0000","last_active":"2024-09-19T21:34:57.938597+0000","last_peered":"2024-09-19T21:34:57.938597+0000","last_clean":"2024-09-19T21:34:57.938597+0000","last_became_active":"2024-09-19T21:34:30.002669+0000","last_became_peered":"2024-09-19T21:34:30.002669+0000","last_unstale":"2024-09-19T21:34:57.938597+0000","last_undegraded":"2024-09-19T21:34:57.938597+0000","last_fullsized":"2024-09-19T21:34:57.938597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:54:28.569315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":10300947,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":1616,"num_read_kb":15177,"num_write":60,"num_write_kb":539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"44'45","reported_seq":398,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.505881+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:34:57.505881+0000","last_peered":"2024-09-19T21:34:57.505881+0000","last_clean":"2024-09-19T21:34:57.505881+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:34:57.505881+0000","last_undegraded":"2024-09-19T21:34:57.505881+0000","last_fullsized":"2024-09-19T21:34:57.505881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":17823820,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":355,"num_read_kb":3607,"num_write":36,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"43'79","reported_seq":3781,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.743249+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:34:56.743249+0000","last_peered":"2024-09-19T21:34:56.743249+0000","last_clean":"2024-09-19T21:34:56.743249+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:34:56.743249+0000","last_undegraded":"2024-09-19T21:34:56.743249+0000","last_fullsized":"2024-09-19T21:34:56.743249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":10688046,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":4178,"num_read_kb":10410,"num_write":47,"num_write_kb":282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"44'30","reported_seq":134,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.347576+0000","last_change":"2024-09-19T21:34:32.012032+0000","last_active":"2024-09-19T21:34:57.347576+0000","last_peered":"2024-09-19T21:34:57.347576+0000","last_clean":"2024-09-19T21:34:57.347576+0000","last_became_active":"2024-09-19T21:34:30.011066+0000","last_became_peered":"2024-09-19T21:34:30.011066+0000","last_unstale":"2024-09-19T21:34:57.347576+0000","last_undegraded":"2024-09-19T21:34:57.347576+0000","last_fullsized":"2024-09-19T21:34:57.347576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:40:19.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":2167296,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":78,"num_read_kb":1345,"num_write":24,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"44'29","reported_seq":153,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.864537+0000","last_change":"2024-09-19T21:34:32.024453+0000","last_active":"2024-09-19T21:34:57.864537+0000","last_peered":"2024-09-19T21:34:57.864537+0000","last_clean":"2024-09-19T21:34:57.864537+0000","last_became_active":"2024-09-19T21:34:30.006875+0000","last_became_peered":"2024-09-19T21:34:30.006875+0000","last_unstale":"2024-09-19T21:34:57.864537+0000","last_undegraded":"2024-09-19T21:34:57.864537+0000","last_fullsized":"2024-09-19T21:34:57.864537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:26:13.507158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":3570216,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":113,"num_read_kb":1388,"num_write":24,"num_write_kb":183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"42'30","reported_seq":191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.275569+0000","last_change":"2024-09-19T21:34:32.006201+0000","last_active":"2024-09-19T21:34:56.275569+0000","last_peered":"2024-09-19T21:34:56.275569+0000","last_clean":"2024-09-19T21:34:56.275569+0000","last_became_active":"2024-09-19T21:34:30.008159+0000","last_became_peered":"2024-09-19T21:34:30.008159+0000","last_unstale":"2024-09-19T21:34:56.275569+0000","last_undegraded":"2024-09-19T21:34:56.275569+0000","last_fullsized":"2024-09-19T21:34:56.275569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:44:57.994848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":1310871,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":142,"num_read_kb":1344,"num_write":25,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"42'49","reported_seq":2212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259496+0000","last_change":"2024-09-19T21:34:32.002842+0000","last_active":"2024-09-19T21:34:56.259496+0000","last_peered":"2024-09-19T21:34:56.259496+0000","last_clean":"2024-09-19T21:34:56.259496+0000","last_became_active":"2024-09-19T21:34:30.010116+0000","last_became_peered":"2024-09-19T21:34:30.010116+0000","last_unstale":"2024-09-19T21:34:56.259496+0000","last_undegraded":"2024-09-19T21:34:56.259496+0000","last_fullsized":"2024-09-19T21:34:56.259496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:38:12.596445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":18199568,"num_objects":17,"num_object_clones":7,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":2163,"num_read_kb":7872,"num_write":45,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259534+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:34:56.259534+0000","last_peered":"2024-09-19T21:34:56.259534+0000","last_clean":"2024-09-19T21:34:56.259534+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:34:56.259534+0000","last_undegraded":"2024-09-19T21:34:56.259534+0000","last_fullsized":"2024-09-19T21:34:56.259534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"44'148","reported_seq":1345,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.536785+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:34:57.536785+0000","last_peered":"2024-09-19T21:34:57.536785+0000","last_clean":"2024-09-19T21:34:57.536785+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:34:57.536785+0000","last_undegraded":"2024-09-19T21:34:57.536785+0000","last_fullsized":"2024-09-19T21:34:57.536785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":41410085,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":1578,"num_read_kb":7898,"num_write":140,"num_write_kb":2951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.268461+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:34:56.268461+0000","last_peered":"2024-09-19T21:34:56.268461+0000","last_clean":"2024-09-19T21:34:56.268461+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:34:56.268461+0000","last_undegraded":"2024-09-19T21:34:56.268461+0000","last_fullsized":"2024-09-19T21:34:56.268461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"42'193","reported_seq":898,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.268574+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:34:56.268574+0000","last_peered":"2024-09-19T21:34:56.268574+0000","last_clean":"2024-09-19T21:34:56.268574+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:34:56.268574+0000","last_undegraded":"2024-09-19T21:34:56.268574+0000","last_fullsized":"2024-09-19T21:34:56.268574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":1900574,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":1553,"num_read_kb":3097,"num_write":325,"num_write_kb":814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259320+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:34:56.259320+0000","last_peered":"2024-09-19T21:34:56.259320+0000","last_clean":"2024-09-19T21:34:56.259320+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:34:56.259320+0000","last_undegraded":"2024-09-19T21:34:56.259320+0000","last_fullsized":"2024-09-19T21:34:56.259320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"44'59","reported_seq":822,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.951533+0000","last_change":"2024-09-19T21:34:32.024327+0000","last_active":"2024-09-19T21:34:57.951533+0000","last_peered":"2024-09-19T21:34:57.951533+0000","last_clean":"2024-09-19T21:34:57.951533+0000","last_became_active":"2024-09-19T21:34:30.007041+0000","last_became_peered":"2024-09-19T21:34:30.007041+0000","last_unstale":"2024-09-19T21:34:57.951533+0000","last_undegraded":"2024-09-19T21:34:57.951533+0000","last_fullsized":"2024-09-19T21:34:57.951533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:31:26.974439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":11622400,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":754,"num_read_kb":5458,"num_write":47,"num_write_kb":1685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T21:34:54.140344+0000","last_change":"2024-09-19T21:34:31.005184+0000","last_active":"2024-09-19T21:34:54.140344+0000","last_peered":"2024-09-19T21:34:54.140344+0000","last_clean":"2024-09-19T21:34:54.140344+0000","last_became_active":"2024-09-19T21:34:31.004633+0000","last_became_peered":"2024-09-19T21:34:31.004633+0000","last_unstale":"2024-09-19T21:34:54.140344+0000","last_undegraded":"2024-09-19T21:34:54.140344+0000","last_fullsized":"2024-09-19T21:34:54.140344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:21.400292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"44'41","reported_seq":1015,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.680152+0000","last_change":"2024-09-19T21:34:32.024762+0000","last_active":"2024-09-19T21:34:57.680152+0000","last_peered":"2024-09-19T21:34:57.680152+0000","last_clean":"2024-09-19T21:34:57.680152+0000","last_became_active":"2024-09-19T21:34:30.006903+0000","last_became_peered":"2024-09-19T21:34:30.006903+0000","last_unstale":"2024-09-19T21:34:57.680152+0000","last_undegraded":"2024-09-19T21:34:57.680152+0000","last_fullsized":"2024-09-19T21:34:57.680152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.226419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":16861216,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":975,"num_read_kb":3930,"num_write":36,"num_write_kb":2326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"42'205","reported_seq":1059,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.963424+0000","last_change":"2024-09-19T21:34:32.006802+0000","last_active":"2024-09-19T21:34:56.963424+0000","last_peered":"2024-09-19T21:34:56.963424+0000","last_clean":"2024-09-19T21:34:56.963424+0000","last_became_active":"2024-09-19T21:34:30.011547+0000","last_became_peered":"2024-09-19T21:34:30.011547+0000","last_unstale":"2024-09-19T21:34:56.963424+0000","last_undegraded":"2024-09-19T21:34:56.963424+0000","last_fullsized":"2024-09-19T21:34:56.963424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:45:47.281981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":1749078,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":2302,"num_read_kb":2922,"num_write":249,"num_write_kb":544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":41,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.275436+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:34:56.275436+0000","last_peered":"2024-09-19T21:34:56.275436+0000","last_clean":"2024-09-19T21:34:56.275436+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:34:56.275436+0000","last_undegraded":"2024-09-19T21:34:56.275436+0000","last_fullsized":"2024-09-19T21:34:56.275436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"43'55","reported_seq":1559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.182877+0000","last_change":"2024-09-19T21:34:32.011746+0000","last_active":"2024-09-19T21:34:57.182877+0000","last_peered":"2024-09-19T21:34:57.182877+0000","last_clean":"2024-09-19T21:34:57.182877+0000","last_became_active":"2024-09-19T21:34:30.013347+0000","last_became_peered":"2024-09-19T21:34:30.013347+0000","last_unstale":"2024-09-19T21:34:57.182877+0000","last_undegraded":"2024-09-19T21:34:57.182877+0000","last_fullsized":"2024-09-19T21:34:57.182877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.129809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":20838980,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1512,"num_read_kb":27868,"num_write":69,"num_write_kb":3723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T21:34:54.140228+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:34:54.140228+0000","last_peered":"2024-09-19T21:34:54.140228+0000","last_clean":"2024-09-19T21:34:54.140228+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:34:54.140228+0000","last_undegraded":"2024-09-19T21:34:54.140228+0000","last_fullsized":"2024-09-19T21:34:54.140228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"44'85","reported_seq":766,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.867052+0000","last_change":"2024-09-19T21:34:32.024303+0000","last_active":"2024-09-19T21:34:57.867052+0000","last_peered":"2024-09-19T21:34:57.867052+0000","last_clean":"2024-09-19T21:34:57.867052+0000","last_became_active":"2024-09-19T21:34:30.009730+0000","last_became_peered":"2024-09-19T21:34:30.009730+0000","last_unstale":"2024-09-19T21:34:57.867052+0000","last_undegraded":"2024-09-19T21:34:57.867052+0000","last_fullsized":"2024-09-19T21:34:57.867052+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":1162240,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":1308,"num_read_kb":2575,"num_write":48,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259767+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:34:56.259767+0000","last_peered":"2024-09-19T21:34:56.259767+0000","last_clean":"2024-09-19T21:34:56.259767+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:34:56.259767+0000","last_undegraded":"2024-09-19T21:34:56.259767+0000","last_fullsized":"2024-09-19T21:34:56.259767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"42'75","reported_seq":1509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.790245+0000","last_change":"2024-09-19T21:34:32.005130+0000","last_active":"2024-09-19T21:34:56.790245+0000","last_peered":"2024-09-19T21:34:56.790245+0000","last_clean":"2024-09-19T21:34:56.790245+0000","last_became_active":"2024-09-19T21:34:30.009669+0000","last_became_peered":"2024-09-19T21:34:30.009669+0000","last_unstale":"2024-09-19T21:34:56.790245+0000","last_undegraded":"2024-09-19T21:34:56.790245+0000","last_fullsized":"2024-09-19T21:34:56.790245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:20:42.350074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":41026613,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1865,"num_read_kb":14085,"num_write":55,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":40,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259379+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:34:56.259379+0000","last_peered":"2024-09-19T21:34:56.259379+0000","last_clean":"2024-09-19T21:34:56.259379+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:34:56.259379+0000","last_undegraded":"2024-09-19T21:34:56.259379+0000","last_fullsized":"2024-09-19T21:34:56.259379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.259200+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:34:56.259200+0000","last_peered":"2024-09-19T21:34:56.259200+0000","last_clean":"2024-09-19T21:34:56.259200+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:34:56.259200+0000","last_undegraded":"2024-09-19T21:34:56.259200+0000","last_fullsized":"2024-09-19T21:34:56.259200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"42'97","reported_seq":4351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.174385+0000","last_change":"2024-09-19T21:34:32.024996+0000","last_active":"2024-09-19T21:34:57.174385+0000","last_peered":"2024-09-19T21:34:57.174385+0000","last_clean":"2024-09-19T21:34:57.174385+0000","last_became_active":"2024-09-19T21:34:30.009636+0000","last_became_peered":"2024-09-19T21:34:30.009636+0000","last_unstale":"2024-09-19T21:34:57.174385+0000","last_undegraded":"2024-09-19T21:34:57.174385+0000","last_fullsized":"2024-09-19T21:34:57.174385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:11.666768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":46722064,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":4070,"num_read_kb":20720,"num_write":71,"num_write_kb":709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"42'57","reported_seq":785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.275324+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:34:56.275324+0000","last_peered":"2024-09-19T21:34:56.275324+0000","last_clean":"2024-09-19T21:34:56.275324+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:34:56.275324+0000","last_undegraded":"2024-09-19T21:34:56.275324+0000","last_fullsized":"2024-09-19T21:34:56.275324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":12253696,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":726,"num_read_kb":4469,"num_write":34,"num_write_kb":2002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"44'103","reported_seq":1114,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.964534+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:34:57.964534+0000","last_peered":"2024-09-19T21:34:57.964534+0000","last_clean":"2024-09-19T21:34:57.964534+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:34:57.964534+0000","last_undegraded":"2024-09-19T21:34:57.964534+0000","last_fullsized":"2024-09-19T21:34:57.964534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":5785126,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1259,"num_read_kb":1441,"num_write":130,"num_write_kb":419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"41'19","reported_seq":157,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.796354+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:34:56.796354+0000","last_peered":"2024-09-19T21:34:56.796354+0000","last_clean":"2024-09-19T21:34:56.796354+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:34:56.796354+0000","last_undegraded":"2024-09-19T21:34:56.796354+0000","last_fullsized":"2024-09-19T21:34:56.796354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":2955791,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":116,"num_read_kb":1490,"num_write":16,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"42'17","reported_seq":213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:56.275385+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:34:56.275385+0000","last_peered":"2024-09-19T21:34:56.275385+0000","last_clean":"2024-09-19T21:34:56.275385+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:34:56.275385+0000","last_undegraded":"2024-09-19T21:34:56.275385+0000","last_fullsized":"2024-09-19T21:34:56.275385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":1681920,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":166,"num_read_kb":305,"num_write":14,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"44'48","reported_seq":279,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.611129+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:34:57.611129+0000","last_peered":"2024-09-19T21:34:57.611129+0000","last_clean":"2024-09-19T21:34:57.611129+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:34:57.611129+0000","last_undegraded":"2024-09-19T21:34:57.611129+0000","last_fullsized":"2024-09-19T21:34:57.611129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":17975296,"num_objects":21,"num_object_clones":9,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":226,"num_read_kb":2323,"num_write":34,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"42'59","reported_seq":1604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.181553+0000","last_change":"2024-09-19T21:34:32.024576+0000","last_active":"2024-09-19T21:34:57.181553+0000","last_peered":"2024-09-19T21:34:57.181553+0000","last_clean":"2024-09-19T21:34:57.181553+0000","last_became_active":"2024-09-19T21:34:30.006185+0000","last_became_peered":"2024-09-19T21:34:30.006185+0000","last_unstale":"2024-09-19T21:34:57.181553+0000","last_undegraded":"2024-09-19T21:34:57.181553+0000","last_fullsized":"2024-09-19T21:34:57.181553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:08:25.609727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":24332800,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1543,"num_read_kb":30472,"num_write":77,"num_write_kb":3893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"44'278","reported_seq":674,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.964623+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:34:57.964623+0000","last_peered":"2024-09-19T21:34:57.964623+0000","last_clean":"2024-09-19T21:34:57.964623+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:34:57.964623+0000","last_undegraded":"2024-09-19T21:34:57.964623+0000","last_fullsized":"2024-09-19T21:34:57.964623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":2217639,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1705,"num_read_kb":2416,"num_write":466,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"44'136","reported_seq":1538,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.950385+0000","last_change":"2024-09-19T21:34:32.000772+0000","last_active":"2024-09-19T21:34:57.950385+0000","last_peered":"2024-09-19T21:34:57.950385+0000","last_clean":"2024-09-19T21:34:57.950385+0000","last_became_active":"2024-09-19T21:34:30.007945+0000","last_became_peered":"2024-09-19T21:34:30.007945+0000","last_unstale":"2024-09-19T21:34:57.950385+0000","last_undegraded":"2024-09-19T21:34:57.950385+0000","last_fullsized":"2024-09-19T21:34:57.950385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:20:07.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":15759360,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":2301,"num_read_kb":4157,"num_write":89,"num_write_kb":406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"44'94","reported_seq":1018,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.956796+0000","last_change":"2024-09-19T21:34:32.023713+0000","last_active":"2024-09-19T21:34:57.956796+0000","last_peered":"2024-09-19T21:34:57.956796+0000","last_clean":"2024-09-19T21:34:57.956796+0000","last_became_active":"2024-09-19T21:34:30.003969+0000","last_became_peered":"2024-09-19T21:34:30.003969+0000","last_unstale":"2024-09-19T21:34:57.956796+0000","last_undegraded":"2024-09-19T21:34:57.956796+0000","last_fullsized":"2024-09-19T21:34:57.956796+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:36.748615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":48013312,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1216,"num_read_kb":4086,"num_write":64,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"42'71","reported_seq":3930,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.183328+0000","last_change":"2024-09-19T21:34:32.011754+0000","last_active":"2024-09-19T21:34:57.183328+0000","last_peered":"2024-09-19T21:34:57.183328+0000","last_clean":"2024-09-19T21:34:57.183328+0000","last_became_active":"2024-09-19T21:34:30.012126+0000","last_became_peered":"2024-09-19T21:34:30.012126+0000","last_unstale":"2024-09-19T21:34:57.183328+0000","last_undegraded":"2024-09-19T21:34:57.183328+0000","last_fullsized":"2024-09-19T21:34:57.183328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:32.355336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":53050368,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":3883,"num_read_kb":48479,"num_write":78,"num_write_kb":3823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"44'88","reported_seq":1191,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-19T21:34:57.610463+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:34:57.610463+0000","last_peered":"2024-09-19T21:34:57.610463+0000","last_clean":"2024-09-19T21:34:57.610463+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:34:57.610463+0000","last_undegraded":"2024-09-19T21:34:57.610463+0000","last_fullsized":"2024-09-19T21:34:57.610463+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":23816780,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1131,"num_read_kb":6052,"num_write":67,"num_write_kb":1780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":622019877,"num_objects":464,"num_object_clones":144,"num_object_copies":928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":464,"num_whiteouts":177,"num_read":49858,"num_read_kb":269600,"num_write":2997,"num_write_kb":59384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77389824,"data_stored":77050442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2936,"ondisk_log_size":2936,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51584,"kb_used_data":20472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94320256,"statfs":{"total":96636764160,"available":96583942144,"internally_reserved":0,"allocated":20963328,"data_stored":20570645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36392,"kb_used_data":5276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335448,"statfs":{"total":96636764160,"available":96599498752,"internally_reserved":0,"allocated":5402624,"data_stored":5134086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43888,"kb_used_data":12648,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327952,"statfs":{"total":96636764160,"available":96591822848,"internally_reserved":0,"allocated":12951552,"data_stored":12709126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":19804,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":20279296,"data_stored":19887919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50540,"kb_used_data":15332,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94321300,"statfs":{"total":96636764160,"available":96585011200,"internally_reserved":0,"allocated":15699968,"data_stored":15271598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":3,"up_from":12,"seq":51539607564,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":5944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":6086656,"data_stored":5700249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15216640,"data_stored":15087408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19206144,"data_stored":19113380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12451840,"data_stored":12507547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5619712,"data_stored":5521011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4943872,"data_stored":4966605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19951616,"data_stored":19854491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:34:59.547 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:34:59.541+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:34:59.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:34:59.645 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:34:59.641+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:34:59.662 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.663 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:59.663 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:34:59.663 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:59.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:34:59.686 INFO:teuthology.orchestra.run.smithi122.stdout:1213 read 0x3a39800 thru 0x3a437a4 (0x9fa5 bytes) 2024-09-19T21:34:59.702 INFO:teuthology.orchestra.run.smithi122.stdout:1214 trunc from 0x883b600 to 0x5c62c00 2024-09-19T21:34:59.743 INFO:teuthology.orchestra.run.smithi122.stdout:1215 read 0x4bc9c00 thru 0x4bd59be (0xbdbf bytes) 2024-09-19T21:34:59.745 INFO:teuthology.orchestra.run.smithi122.stdout:1216 write 0x2e61000 thru 0x2e651e6 (0x41e7 bytes) 2024-09-19T21:34:59.746 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:59.741+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:59.746 INFO:teuthology.orchestra.run.smithi122.stdout:1218 write 0x3511a00 thru 0x3518afb (0x70fc bytes) 2024-09-19T21:34:59.747 INFO:teuthology.orchestra.run.smithi122.stdout:1222 read 0x54a4800 thru 0x54ad084 (0x8885 bytes) 2024-09-19T21:34:59.747 INFO:teuthology.orchestra.run.smithi122.stdout:1224 write 0x4421800 thru 0x4430d7c (0xf57d bytes) 2024-09-19T21:34:59.747 INFO:teuthology.orchestra.run.smithi122.stdout:1225 read 0x1d5f800 thru 0x1d643bb (0x4bbc bytes) 2024-09-19T21:34:59.797 INFO:teuthology.orchestra.run.smithi122.stdout:1227 write 0x94cfa00 thru 0x94cff6a (0x56b bytes) 2024-09-19T21:34:59.799 INFO:teuthology.orchestra.run.smithi122.stdout:1230 punch from 0x5559a00 to 0x5564600, (0xac00 bytes) 2024-09-19T21:34:59.829 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.829 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:34:59.829 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:34:59.829 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:34:59.829 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:34:59.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:34:59.843 INFO:teuthology.orchestra.run.smithi122.stdout:1232 trunc from 0x94d0000 to 0x76fc00 2024-09-19T21:34:59.884 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:34:59.881+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:34:59.910 INFO:teuthology.orchestra.run.smithi122.stdout:1233 trunc from 0x76fc00 to 0x8014e00 2024-09-19T21:34:59.932 INFO:teuthology.orchestra.run.smithi122.stdout:1234 write 0x73bba00 thru 0x73c9a90 (0xe091 bytes) 2024-09-19T21:34:59.933 INFO:teuthology.orchestra.run.smithi122.stdout:1236 read 0x1b1d200 thru 0x1b260e4 (0x8ee5 bytes) 2024-09-19T21:34:59.933 INFO:teuthology.orchestra.run.smithi122.stdout:1237 read 0x530a600 thru 0x53173dc (0xcddd bytes) 2024-09-19T21:34:59.934 INFO:teuthology.orchestra.run.smithi122.stdout:1238 write 0x372dc00 thru 0x373b712 (0xdb13 bytes) 2024-09-19T21:34:59.947 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:34:59.941+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:59.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.545007314999999, 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:34:59.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.544987008, 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:34:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.544967545999999, 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:34:59.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.544933234999998, 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:34:59.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.544907734999999, 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.970 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540954505999999, 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540917382, 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:34:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:34:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540880163000001, 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:34:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:34:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:34:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:34:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540835774000001, 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540818087000002, 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540793137000001, 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:34:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:34:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:34:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:35:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540753846000001, 2024-09-19T21:35:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:35:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540716443000001, 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.540698614, 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:35:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:35:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.537049836000001, 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.537025291999999, 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.537006677000001, 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:35:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:35:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.536957270999999, 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:35:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.536938873, 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:35:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.536913104, 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:35:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.043 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:00.044 INFO:teuthology.orchestra.run.smithi122.stdout:1240 write 0xa02fc00 thru 0xa033363 (0x3764 bytes) 2024-09-19T21:35:00.045 INFO:teuthology.orchestra.run.smithi122.stdout:1241 write 0x4c2000 thru 0x4ca69e (0x869f bytes) 2024-09-19T21:35:00.045 INFO:teuthology.orchestra.run.smithi122.stdout:1247 read 0x55d3600 thru 0x55e0058 (0xca59 bytes) 2024-09-19T21:35:00.045 INFO:teuthology.orchestra.run.smithi122.stdout:1248 write 0x7fcc800 thru 0x7fcd851 (0x1052 bytes) 2024-09-19T21:35:00.045 INFO:teuthology.orchestra.run.smithi122.stdout:1249 clone 11 order 25 su 1048576 sc 6 2024-09-19T21:35:00.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:00.048 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:00.045+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:00.149 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:00.145+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:00.167 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:00.167 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:00.167 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:00.167 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:00.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:00.249 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:00.245+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:00.300 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:35:00.334 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:00.335 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:00.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:00.350 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:00.345+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:00.451 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:00.445+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1075 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 7168~512] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7400757890000005, 2024-09-19T21:35:00.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030194739000000002, 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1075 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712126+0000", 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5739999999999998e-06 2024-09-19T21:35:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712134+0000", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6550000000000004e-06 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712139+0000", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2270000000000001e-06 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742210+0000", 2024-09-19T21:35:00.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030070403999999998 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742253+0000", 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3010000000000003e-05 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742319+0000", 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6241000000000005e-05 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1077 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7400551020000004, 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030330979000000001, 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1077 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712144+0000", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712148+0000", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3330000000000001e-06 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712156+0000", 2024-09-19T21:35:00.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3280000000000006e-06 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712161+0000", 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2299999999999999e-06 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742353+0000", 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030191913000000001 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742398+0000", 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4669000000000002e-05 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742476+0000", 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8181000000000002e-05 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1480 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7199998970000001, 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030120475000000001, 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1480 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732201+0000", 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4160000000000001e-06 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732205+0000", 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2169999999999996e-06 2024-09-19T21:35:00.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732208+0000", 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6259999999999999e-06 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762084+0000", 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029876117000000001 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762193+0000", 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108906 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762321+0000", 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012741400000000001 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1485 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.470 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7199895859999996, 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030359516999999999, 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1485 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732212+0000", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5269999999999999e-06 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732215+0000", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.692e-06 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732218+0000", 2024-09-19T21:35:00.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6010000000000002e-06 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762393+0000", 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030175415000000001 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762470+0000", 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6774999999999996e-05 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762570+0000", 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9883000000000006e-05 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1540 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7159382460000003, 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.03012836, 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1540 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736261+0000", 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736264+0000", 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3120000000000001e-06 2024-09-19T21:35:00.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736269+0000", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736274+0000", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.69e-06 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766159+0000", 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029884984 2024-09-19T21:35:00.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766269+0000", 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110098 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766390+0000", 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121489 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1543 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7159212479999999, 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030340459, 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1543 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736278+0000", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736281+0000", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4250000000000001e-06 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736286+0000", 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.665e-06 2024-09-19T21:35:00.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736293+0000", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5039999999999999e-06 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766437+0000", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030144388000000001 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766523+0000", 2024-09-19T21:35:00.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5575000000000007e-05 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766619+0000", 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6299999999999996e-05 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1546 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7159009990000005, 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030580726999999999, 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1546 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736300+0000", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8369999999999999e-06 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736304+0000", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9890000000000003e-06 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736314+0000", 2024-09-19T21:35:00.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2550000000000003e-06 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766650+0000", 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030336609 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766753+0000", 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102413 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766880+0000", 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012705999999999999 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1551 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7158834460000003, 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030830699999999999, 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1551 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:00.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736316+0000", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736322+0000", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8279999999999999e-06 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736325+0000", 2024-09-19T21:35:00.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.348e-06 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736328+0000", 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7020000000000001e-06 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766911+0000", 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030582668 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767012+0000", 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101053 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767147+0000", 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135597 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1554 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7158698570000004, 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031084074, 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1554 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736331+0000", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5719999999999999e-06 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736334+0000", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9050000000000001e-06 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736337+0000", 2024-09-19T21:35:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6429999999999999e-06 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767184+0000", 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030846625999999999 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767286+0000", 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010242399999999999 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767414+0000", 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012813500000000001 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1559 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7158395889999998, 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031279296999999998, 2024-09-19T21:35:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1559 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736362+0000", 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.362e-06 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736364+0000", 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.695e-06 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736367+0000", 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767459+0000", 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031091534000000001 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767540+0000", 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1353000000000005e-05 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767640+0000", 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9806e-05 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1562 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.715829651, 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031533528999999998, 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1562 2024-09-19T21:35:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736371+0000", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.373e-06 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736374+0000", 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8480000000000001e-06 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736378+0000", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8850000000000001e-06 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767680+0000", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031301571 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767772+0000", 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2255000000000004e-05 2024-09-19T21:35:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767904+0000", 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181299999999999 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1569 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7157892500000003, 2024-09-19T21:35:00.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031758058999999998, 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1569 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736410+0000", 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736413+0000", 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6960000000000001e-06 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736421+0000", 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6399999999999997e-06 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736425+0000", 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5279999999999997e-06 2024-09-19T21:35:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767935+0000", 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031509713000000002 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768041+0000", 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106397 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768169+0000", 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000127479 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1571 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7157434870000001, 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031967883000000002, 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1571 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736459+0000", 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736466+0000", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5250000000000002e-06 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736470+0000", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768196+0000", 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031726035 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768298+0000", 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010195400000000001 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768424+0000", 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012614899999999999 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1572 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7157251230000004, 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032212024999999998, 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1572 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736474+0000", 2024-09-19T21:35:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736477+0000", 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4940000000000002e-06 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736482+0000", 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5010000000000004e-06 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736488+0000", 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1079999999999998e-06 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768452+0000", 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031963671999999999 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768557+0000", 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104411 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768687+0000", 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000130307 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1574 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7157073799999996, 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032396917999999997, 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1574 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736492+0000", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736495+0000", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3640000000000002e-06 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736500+0000", 2024-09-19T21:35:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7069999999999997e-06 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736505+0000", 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5959999999999997e-06 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768731+0000", 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032225666 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768798+0000", 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7187000000000006e-05 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768890+0000", 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1730000000000004e-05 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1575 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7156900339999996, 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032552508000000001, 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1575 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736512+0000", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736518+0000", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736523+0000", 2024-09-19T21:35:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5670000000000002e-06 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768914+0000", 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032390713000000002 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768976+0000", 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2390000000000004e-05 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769062+0000", 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6248000000000005e-05 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1578 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7156383340000003, 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032774279000000003, 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1578 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:00.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736561+0000", 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736565+0000", 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.557e-06 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736572+0000", 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9419999999999998e-06 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736576+0000", 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4730000000000002e-06 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769102+0000", 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032525674999999997 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769205+0000", 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010330700000000001 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769336+0000", 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013078699999999999 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1579 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.534 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7156191490000001, 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033004987999999999, 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1579 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736580+0000", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736583+0000", 2024-09-19T21:35:00.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8090000000000001e-06 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736589+0000", 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736593+0000", 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5210000000000004e-06 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769367+0000", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032773434999999997 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769463+0000", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6069000000000005e-05 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769586+0000", 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012306200000000001 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1582 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7155380930000002, 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033168969999999999, 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1582 2024-09-19T21:35:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736661+0000", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736663+0000", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.872e-06 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736668+0000", 2024-09-19T21:35:00.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9280000000000001e-06 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736677+0000", 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2709999999999992e-06 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769651+0000", 2024-09-19T21:35:00.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032974366999999997 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769726+0000", 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5244e-05 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769831+0000", 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104737 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.7035781610000003, 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:00.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:00.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:00.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:00.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:00.550 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone10 from 0xa033400 (overlap 0x76fc00) to 0xbe600 2024-09-19T21:35:00.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:00.553 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:00.549+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:00.654 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:00.649+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:00.676 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:00.677 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:00.677 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:00.677 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:00.687 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:00.757 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:00.753+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:00.793 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:00.793 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:00.794 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:00.794 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:00.794 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:00.804 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:00.858 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:00.853+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:00.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:00.928 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:00.928 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.666198192, 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:00.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:35:00.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.933 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.664871723000001, 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.934 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:35:00.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:00.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.664840249000001, 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:35:00.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.942 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.664801295, 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:35:00.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.947 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.664760252000001, 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:35:00.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.952 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655507868000001, 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:00.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:35:00.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655497639, 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:35:00.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655488197, 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:35:00.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655469756999999, 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:35:00.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.973 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655459148999999, 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:35:00.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.65545041, 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:35:00.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:35:00.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655438160999999, 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:35:00.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:35:00.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655239346999998, 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:35:00.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655221483999998, 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:00.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655204938000001, 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:35:00.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:00.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:35:01.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655188592999998, 2024-09-19T21:35:01.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:01.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:35:01.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655171716000002, 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:01.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:01.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:35:01.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655153529, 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:01.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:35:01.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:01.016 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.655135000000001, 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:01.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:35:01.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:35:01.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4809.0:1592 3.1d e42/23 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head v 42'106, mlcod=42'104)", 2024-09-19T21:35:01.021 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.1719815750000002, 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033307211000000003, 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751782+0000", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758281+0000", 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0064992369999999997 2024-09-19T21:35:01.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758315+0000", 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4561999999999999e-05 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758330+0000", 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4822e-05 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758424+0000", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3377999999999999e-05 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758462+0000", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8732000000000001e-05 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785018+0000", 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.026556059999999999 2024-09-19T21:35:01.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785090+0000", 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1971999999999995e-05 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:01.026 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:01.028 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:01.028 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:01.021+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:01.031 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:01.057 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:01.078 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:01.106 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-19T21:35:01.129 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:01.125+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:01.140 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.140 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:01.209 INFO:teuthology.orchestra.run.smithi122.stdout:1256 read 0x1c92a00 thru 0x1c9da95 (0xb096 bytes) 2024-09-19T21:35:01.229 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:01.225+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:01.251 INFO:teuthology.orchestra.run.smithi122.stdout:1257 write 0xb7e4e00 thru 0xb7efe29 (0xb02a bytes) 2024-09-19T21:35:01.279 INFO:teuthology.orchestra.run.smithi122.stdout:1260 trunc from 0xb7f0000 to 0x6f3f200 2024-09-19T21:35:01.307 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:01.316 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.316 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:01.330 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:01.325+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:01.335 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:01.338 INFO:teuthology.orchestra.run.smithi122.stdout:1263 write 0x3c18200 thru 0x3c24ec8 (0xccc9 bytes) 2024-09-19T21:35:01.360 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:01.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:01.382 INFO:teuthology.orchestra.run.smithi122.stdout:1264 trunc from 0x6f3f200 to 0x1f3c800 2024-09-19T21:35:01.431 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:01.425+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:01.435 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.435 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.448 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:01.448 INFO:teuthology.orchestra.run.smithi122.stdout:1265 write 0x3dc0400 thru 0x3dcd169 (0xcd6a bytes) 2024-09-19T21:35:01.460 INFO:teuthology.orchestra.run.smithi122.stdout:1266 trunc from 0x3dcd200 to 0x6ee9c00 2024-09-19T21:35:01.483 INFO:teuthology.orchestra.run.smithi122.stdout:1267 read 0x5378a00 thru 0x5387cc2 (0xf2c3 bytes) 2024-09-19T21:35:01.486 INFO:teuthology.orchestra.run.smithi122.stdout:1270 write 0x3136a00 thru 0x313d45b (0x6a5c bytes) 2024-09-19T21:35:01.486 INFO:teuthology.orchestra.run.smithi122.stdout:1271 read 0x1c1b600 thru 0x1c257fa (0xa1fb bytes) 2024-09-19T21:35:01.487 INFO:teuthology.orchestra.run.smithi122.stdout:1272 punch from 0x377da00 to 0x3788400, (0xaa00 bytes) 2024-09-19T21:35:01.498 INFO:teuthology.orchestra.run.smithi122.stdout:1276 trunc from 0x6ee9c00 to 0xae93400 2024-09-19T21:35:01.517 INFO:teuthology.orchestra.run.smithi122.stdout:1277 read 0xa6a9c00 thru 0xa6ac675 (0x2a76 bytes) 2024-09-19T21:35:01.519 INFO:teuthology.orchestra.run.smithi122.stdout:1282 punch from 0x94ea600 to 0x94f0c00, (0x6600 bytes) 2024-09-19T21:35:01.519 INFO:teuthology.orchestra.run.smithi122.stdout:1283 read 0x9ed3800 thru 0x9edaf5f (0x7760 bytes) 2024-09-19T21:35:01.520 INFO:teuthology.orchestra.run.smithi122.stdout:1284 write 0x3deb200 thru 0x3df1a22 (0x6823 bytes) 2024-09-19T21:35:01.532 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:01.525+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:01.543 INFO:teuthology.orchestra.run.smithi122.stdout:1285 write 0xb638000 thru 0xb63d633 (0x5634 bytes) 2024-09-19T21:35:01.545 INFO:teuthology.orchestra.run.smithi122.stdout:1286 read 0x47c7600 thru 0x47cfeae (0x88af bytes) 2024-09-19T21:35:01.546 INFO:teuthology.orchestra.run.smithi122.stdout:1288 read 0xa339400 thru 0xa33a51a (0x111b bytes) 2024-09-19T21:35:01.547 INFO:teuthology.orchestra.run.smithi122.stdout:1289 read 0x9996400 thru 0x999a43f (0x4040 bytes) 2024-09-19T21:35:01.561 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.561 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:01.587 INFO:teuthology.orchestra.run.smithi122.stdout:1290 write 0xd125400 thru 0xd125d2f (0x930 bytes) 2024-09-19T21:35:01.617 INFO:teuthology.orchestra.run.smithi122.stdout:1292 write 0xda10600 thru 0xda1d6de (0xd0df bytes) 2024-09-19T21:35:01.619 INFO:teuthology.orchestra.run.smithi122.stdout:1293 write 0x412b200 thru 0x413922e (0xe02f bytes) 2024-09-19T21:35:01.620 INFO:teuthology.orchestra.run.smithi122.stdout:1295 write 0x3059800 thru 0x305f98d (0x618e bytes) 2024-09-19T21:35:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:1296 punch from 0x2300800 to 0x2304e00, (0x4600 bytes) 2024-09-19T21:35:01.624 INFO:teuthology.orchestra.run.smithi122.stdout:1297 read 0x55f2a00 thru 0x56016d6 (0xecd7 bytes) 2024-09-19T21:35:01.633 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:01.629+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:01.636 INFO:teuthology.orchestra.run.smithi122.stdout:1298 trunc from 0xda1d800 to 0x8da8c00 2024-09-19T21:35:01.671 INFO:teuthology.orchestra.run.smithi122.stdout:1300 read 0x4f38e00 thru 0x4f43366 (0xa567 bytes) 2024-09-19T21:35:01.674 INFO:teuthology.orchestra.run.smithi122.stdout:1302 write 0x1fef000 thru 0x1fefcfe (0xcff bytes) 2024-09-19T21:35:01.674 INFO:teuthology.orchestra.run.smithi122.stdout:1303 punch from 0xcace00 to 0xcbaa00, (0xdc00 bytes) 2024-09-19T21:35:01.675 INFO:teuthology.orchestra.run.smithi122.stdout:1304 punch from 0x317fc00 to 0x3181800, (0x1c00 bytes) 2024-09-19T21:35:01.675 INFO:teuthology.orchestra.run.smithi122.stdout:1305 write 0x1456600 thru 0x145c770 (0x6171 bytes) 2024-09-19T21:35:01.677 INFO:teuthology.orchestra.run.smithi122.stdout:1307 punch from 0x45c0600 to 0x45c8600, (0x8000 bytes) 2024-09-19T21:35:01.677 INFO:teuthology.orchestra.run.smithi122.stdout:1308 read 0x3eb8e00 thru 0x3ec2880 (0x9a81 bytes) 2024-09-19T21:35:01.684 INFO:teuthology.orchestra.run.smithi122.stdout:1309 trunc from 0x8da8c00 to 0x77ade00 2024-09-19T21:35:01.690 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.691 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:01.734 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:01.729+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:01.743 INFO:teuthology.orchestra.run.smithi122.stdout:1310 write 0xe7ea200 thru 0xe7f762d (0xd42e bytes) 2024-09-19T21:35:01.744 INFO:teuthology.orchestra.run.smithi122.stdout:1312 read 0x9ca9000 thru 0x9caef11 (0x5f12 bytes) 2024-09-19T21:35:01.749 INFO:teuthology.orchestra.run.smithi122.stdout:1313 write 0x4391400 thru 0x43a1101 (0xfd02 bytes) 2024-09-19T21:35:01.750 INFO:teuthology.orchestra.run.smithi122.stdout:1314 read 0x6cffa00 thru 0x6d0979e (0x9d9f bytes) 2024-09-19T21:35:01.752 INFO:teuthology.orchestra.run.smithi122.stdout:1315 write 0x88a2a00 thru 0x88b12a5 (0xe8a6 bytes) 2024-09-19T21:35:01.753 INFO:teuthology.orchestra.run.smithi122.stdout:1317 punch from 0x32c4400 to 0x32cc200, (0x7e00 bytes) 2024-09-19T21:35:01.771 INFO:teuthology.orchestra.run.smithi122.stdout:1318 trunc from 0xe7f7800 to 0x7996200 2024-09-19T21:35:01.815 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.816 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.826 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:01.828 INFO:teuthology.orchestra.run.smithi122.stdout:1322 write 0xaf95200 thru 0xafa15ec (0xc3ed bytes) 2024-09-19T21:35:01.831 INFO:teuthology.orchestra.run.smithi122.stdout:1323 read 0xf69e00 thru 0xf6a6ca (0x8cb bytes) 2024-09-19T21:35:01.832 INFO:teuthology.orchestra.run.smithi122.stdout:1324 read 0x8349400 thru 0x8350e30 (0x7a31 bytes) 2024-09-19T21:35:01.833 INFO:teuthology.orchestra.run.smithi122.stdout:1325 punch from 0x8f85600 to 0x8f8c800, (0x7200 bytes) 2024-09-19T21:35:01.834 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:01.829+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:01.834 INFO:teuthology.orchestra.run.smithi122.stdout:1327 write 0x5e8c400 thru 0x5e9430c (0x7f0d bytes) 2024-09-19T21:35:01.835 INFO:teuthology.orchestra.run.smithi122.stdout:1328 read 0x4018800 thru 0x401c69f (0x3ea0 bytes) 2024-09-19T21:35:01.836 INFO:teuthology.orchestra.run.smithi122.stdout:1329 read 0x9e94800 thru 0x9e96606 (0x1e07 bytes) 2024-09-19T21:35:01.837 INFO:teuthology.orchestra.run.smithi122.stdout:1330 write 0x2e6a200 thru 0x2e78ee6 (0xece7 bytes) 2024-09-19T21:35:01.839 INFO:teuthology.orchestra.run.smithi122.stdout:1331 write 0x6cf800 thru 0x6da011 (0xa812 bytes) 2024-09-19T21:35:01.846 INFO:teuthology.orchestra.run.smithi122.stdout:1333 trunc from 0xafa1600 to 0xc89b600 2024-09-19T21:35:01.868 INFO:teuthology.orchestra.run.smithi122.stdout:1335 punch from 0x1085400 to 0x108ee00, (0x9a00 bytes) 2024-09-19T21:35:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:1336 punch from 0xda0600 to 0xda2a00, (0x2400 bytes) 2024-09-19T21:35:01.881 INFO:teuthology.orchestra.run.smithi122.stdout:1339 trunc from 0xc89b600 to 0x8350c00 2024-09-19T21:35:01.911 INFO:teuthology.orchestra.run.smithi122.stdout:1341 trunc from 0x8350c00 to 0x8aeee00 2024-09-19T21:35:01.931 INFO:teuthology.orchestra.run.smithi122.stdout:1342 read 0x1a0e000 thru 0x1a19837 (0xb838 bytes) 2024-09-19T21:35:01.935 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:01.929+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:01.945 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:01.945 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:01.946 INFO:teuthology.orchestra.run.smithi122.stdout:1343 trunc from 0x8aeee00 to 0x3bc1600 2024-09-19T21:35:01.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:02.016 INFO:teuthology.orchestra.run.smithi122.stdout:1345 write 0xa1cf400 thru 0xa1d1f4c (0x2b4d bytes) 2024-09-19T21:35:02.018 INFO:teuthology.orchestra.run.smithi122.stdout:1346 write 0x521e800 thru 0x522ba0c (0xd20d bytes) 2024-09-19T21:35:02.018 INFO:teuthology.orchestra.run.smithi122.stdout:1348 clone 12 order 25 su 524288 sc 3 2024-09-19T21:35:02.037 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:02.033+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:02.063 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.063 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:02.138 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:02.133+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:02.180 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.181 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:02.239 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:02.233+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:02.303 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.304 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.314 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:02.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:35:02.339 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:02.333+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:02.420 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.420 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:02.440 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:02.437+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:02.474 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone11 2024-09-19T21:35:02.542 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:02.537+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:02.585 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.586 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:02.642 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:02.637+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:02.707 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.708 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:02.743 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:02.737+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:02.825 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.825 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:02.844 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:02.841+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:02.942 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:02.942 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:02.945 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:02.941+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:02.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:03.046 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:03.041+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:03.067 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:03.067 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:03.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:03.147 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:03.141+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:03.187 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:03.188 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:03.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:03.248 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:03.245+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:03.307 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:03.307 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:03.312 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:35:03.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:03.349 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:03.345+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:03.374 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:03.402 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-19T21:35:03.424 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:03.424 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:03.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:03.450 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:03.445+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:03.512 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:03.533 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:03.535 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:03.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:03.535 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:03.535 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:03.535 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:03.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:03.551 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:03.545+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:03.552 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:03.574 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:03.597 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:03.620 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:03.624 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:03.625 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:03.648 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 47.711968224000003, 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:03.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:03.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4788 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375742495000001, 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053208699999999998, 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4788 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.656 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269024+0000", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269027+0000", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0889999999999998e-06 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269032+0000", 2024-09-19T21:35:03.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.31e-06 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269050+0000", 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8138000000000002e-05 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322058+0000", 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053008161999999998 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322126+0000", 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8269999999999995e-05 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322233+0000", 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107027 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.660 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4789 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 423936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375711923000001, 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053201143999999999, 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4789 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269066+0000", 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1392e-05 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269073+0000", 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0500000000000003e-06 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269079+0000", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322103+0000", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053024486000000003 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322168+0000", 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949000000000004e-05 2024-09-19T21:35:03.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322256+0000", 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8187000000000001e-05 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4790 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375684095, 2024-09-19T21:35:03.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053307251999999999, 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4790 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269097+0000", 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4481000000000001e-05 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0243e-05 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0139999999999998e-06 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322255+0000", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053142366000000003 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322323+0000", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7725999999999995e-05 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322390+0000", 2024-09-19T21:35:03.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7800999999999997e-05 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4791 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375650211, 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053280393000000002, 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.670 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4791 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269119+0000", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269125+0000", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5180000000000004e-06 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269129+0000", 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6990000000000002e-06 2024-09-19T21:35:03.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322279+0000", 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053149890999999998 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322342+0000", 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2392999999999997e-05 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322397+0000", 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5798999999999999e-05 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4792 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375634249000001, 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053414182999999997, 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4792 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269141+0000", 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6810000000000003e-06 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3549999999999998e-06 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322416+0000", 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053270709999999999 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322474+0000", 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8161000000000001e-05 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322547+0000", 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3357999999999998e-05 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4796 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375618600999999, 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053558802000000003, 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4796 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269148+0000", 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0439999999999998e-06 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269155+0000", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.967e-06 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269167+0000", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1935e-05 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322574+0000", 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053407102999999997 2024-09-19T21:35:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322642+0000", 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7683999999999998e-05 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322707+0000", 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5421999999999998e-05 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4799 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375595602000001, 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054302238000000003, 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4799 2024-09-19T21:35:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269171+0000", 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0429999999999999e-06 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269186+0000", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9840000000000001e-06 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323330+0000", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054144172999999997 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323399+0000", 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9543999999999997e-05 2024-09-19T21:35:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323474+0000", 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4564e-05 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4868 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375577047, 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053685682999999998, 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4868 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269192+0000", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2639999999999998e-06 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269198+0000", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8810000000000001e-06 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1049999999999997e-06 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322728+0000", 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053525831000000003 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322792+0000", 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4367000000000002e-05 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322876+0000", 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3558000000000002e-05 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4869 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375561004000001, 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053842947000000002, 2024-09-19T21:35:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4869 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269208+0000", 2024-09-19T21:35:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3879999999999998e-06 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269214+0000", 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269221+0000", 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.657e-06 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322897+0000", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053676662 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322966+0000", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9085999999999996e-05 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323049+0000", 2024-09-19T21:35:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2742000000000002e-05 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4870 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.375357231999999, 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053808917999999997, 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4870 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269225+0000", 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998155 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269413+0000", 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018773800000000001 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269420+0000", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7399999999999998e-06 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269424+0000", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9389999999999999e-06 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323071+0000", 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053647087000000003 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323137+0000", 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5810999999999997e-05 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323219+0000", 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2236000000000003e-05 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4871 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374716255999999, 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053346843999999997, 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4871 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269436+0000", 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9993844 2024-09-19T21:35:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270057+0000", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062110799999999999 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270066+0000", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2709999999999998e-06 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270073+0000", 2024-09-19T21:35:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9770000000000003e-06 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323240+0000", 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053167430000000002 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323312+0000", 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1581000000000005e-05 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323398+0000", 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5939999999999994e-05 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4872 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374687714, 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053465749, 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4872 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270079+0000", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270082+0000", 2024-09-19T21:35:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9730000000000002e-06 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270088+0000", 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270093+0000", 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323425+0000", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053332349000000001 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323493+0000", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7828000000000002e-05 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323545+0000", 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2166000000000002e-05 2024-09-19T21:35:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4873 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 106496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374669901000001, 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053689477999999999, 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4873 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270099+0000", 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270106+0000", 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270111+0000", 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7520000000000004e-06 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323583+0000", 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053471779999999997 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323664+0000", 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1402000000000004e-05 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323787+0000", 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122404 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4874 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 116736~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374642825999999, 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053900483999999999, 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4874 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270131+0000", 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9079999999999998e-06 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270138+0000", 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5699999999999998e-06 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270143+0000", 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323826+0000", 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053683467999999998 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323902+0000", 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5350000000000002e-05 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324025+0000", 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012334600000000001 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4875 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374607062999999, 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054010483999999997, 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4875 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270177+0000", 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6985000000000001e-05 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270188+0000", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0993e-05 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270193+0000", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7679999999999997e-06 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324068+0000", 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053874248 2024-09-19T21:35:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324119+0000", 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1615e-05 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324171+0000", 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440999999999998e-05 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4876 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374322674999998, 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053834246000000002, 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4876 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270198+0000", 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997535 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270450+0000", 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025216000000000003 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270460+0000", 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8409999999999995e-06 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270467+0000", 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6680000000000004e-06 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324197+0000", 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053730214999999998 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324239+0000", 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1755999999999998e-05 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324279+0000", 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0224e-05 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4877 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374293396999999, 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054028259000000002, 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4877 2024-09-19T21:35:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270473+0000", 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270476+0000", 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8590000000000001e-06 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270490+0000", 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4071e-05 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270496+0000", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7599999999999999e-06 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324308+0000", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053811953000000003 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324349+0000", 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0747999999999997e-05 2024-09-19T21:35:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324502+0000", 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015331599999999999 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4878 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.374246049, 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054078566000000002, 2024-09-19T21:35:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4878 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270524+0000", 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271662+0000", 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001138114 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271683+0000", 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0638000000000001e-05 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324520+0000", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052837566000000002 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324559+0000", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8782999999999998e-05 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324600+0000", 2024-09-19T21:35:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.066e-05 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4881 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.372718766999999, 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053152144999999998, 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4881 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272051+0000", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8890000000000002e-06 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272057+0000", 2024-09-19T21:35:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6030000000000004e-06 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272066+0000", 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5540000000000003e-06 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325024+0000", 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052957551999999998 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325114+0000", 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0708e-05 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325201+0000", 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6236999999999994e-05 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:03.747 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:03.748 INFO:teuthology.orchestra.run.smithi122.stdout:1349 punch from 0x2db7c00 to 0x2dc5800, (0xdc00 bytes) 2024-09-19T21:35:03.750 INFO:teuthology.orchestra.run.smithi122.stdout:1350 punch from 0x6186000 to 0x618f400, (0x9400 bytes) 2024-09-19T21:35:03.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:03.751 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:03.745+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:03.779 INFO:teuthology.orchestra.run.smithi122.stdout:1351 write 0xa30fc00 thru 0xa31c71b (0xcb1c bytes) 2024-09-19T21:35:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:1352 write 0x7a37c00 thru 0x7a3ef1c (0x731d bytes) 2024-09-19T21:35:03.782 INFO:teuthology.orchestra.run.smithi122.stdout:1353 write 0x670cc00 thru 0x6719c2a (0xd02b bytes) 2024-09-19T21:35:03.783 INFO:teuthology.orchestra.run.smithi122.stdout:1354 read 0x588a00 thru 0x59122e (0x882f bytes) 2024-09-19T21:35:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:1357 write 0x22e8800 thru 0x22f4821 (0xc022 bytes) 2024-09-19T21:35:03.786 INFO:teuthology.orchestra.run.smithi122.stdout:1362 punch from 0x347d000 to 0x347f400, (0x2400 bytes) 2024-09-19T21:35:03.792 INFO:teuthology.orchestra.run.smithi122.stdout:1369 write 0x41a9c00 thru 0x41b0041 (0x6442 bytes) 2024-09-19T21:35:03.848 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:03.845+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:03.862 INFO:teuthology.orchestra.run.smithi122.stdout:1370 write 0xe8f4e00 thru 0xe900f20 (0xc121 bytes) 2024-09-19T21:35:03.873 INFO:teuthology.orchestra.run.smithi122.stdout:1371 trunc from 0xe901000 to 0xb83a800 2024-09-19T21:35:03.913 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:03.913 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:03.913 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:03.913 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:03.917 INFO:teuthology.orchestra.run.smithi122.stdout:1372 punch from 0xaca3800 to 0xaca6a00, (0x3200 bytes) 2024-09-19T21:35:03.919 INFO:teuthology.orchestra.run.smithi122.stdout:1374 read 0xb235200 thru 0xb2418df (0xc6e0 bytes) 2024-09-19T21:35:03.921 INFO:teuthology.orchestra.run.smithi122.stdout:1375 read 0x8603000 thru 0x860d719 (0xa71a bytes) 2024-09-19T21:35:03.922 INFO:teuthology.orchestra.run.smithi122.stdout:1376 punch from 0x2f95200 to 0x2fa2c00, (0xda00 bytes) 2024-09-19T21:35:03.922 INFO:teuthology.orchestra.run.smithi122.stdout:1379 write 0x82e2c00 thru 0x82eddba (0xb1bb bytes) 2024-09-19T21:35:03.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:03.924 INFO:teuthology.orchestra.run.smithi122.stdout:1381 write 0xb3a5400 thru 0xb3b410e (0xed0f bytes) 2024-09-19T21:35:03.925 INFO:teuthology.orchestra.run.smithi122.stdout:1385 write 0xe58600 thru 0xe5bad3 (0x34d4 bytes) 2024-09-19T21:35:03.949 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:03.945+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:03.979 INFO:teuthology.orchestra.run.smithi122.stdout:1389 trunc from 0xb83a800 to 0x2a68200 2024-09-19T21:35:04.034 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:04.034 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:04.034 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:04.034 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:04.035 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:04.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:04.051 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:04.045+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:04.068 INFO:teuthology.orchestra.run.smithi122.stdout:1390 write 0x8893400 thru 0x889420d (0xe0e bytes) 2024-09-19T21:35:04.070 INFO:teuthology.orchestra.run.smithi122.stdout:1391 punch from 0x4a68400 to 0x4a76c00, (0xe800 bytes) 2024-09-19T21:35:04.086 INFO:teuthology.orchestra.run.smithi122.stdout:1392 trunc from 0x8894400 to 0x2538200 2024-09-19T21:35:04.133 INFO:teuthology.orchestra.run.smithi122.stdout:1393 read 0x157f200 thru 0x1588cf6 (0x9af7 bytes) 2024-09-19T21:35:04.137 INFO:teuthology.orchestra.run.smithi122.stdout:1394 write 0x240bc00 thru 0x24126b6 (0x6ab7 bytes) 2024-09-19T21:35:04.151 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:04.145+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.929302133, 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:04.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1446 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:04.166 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4555322989999997, 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028575401, 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1446 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701217+0000", 2024-09-19T21:35:04.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4450000000000001e-06 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701232+0000", 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5211999999999999e-05 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701236+0000", 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8689999999999997e-06 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729646+0000", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028410086000000001 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729707+0000", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0690999999999999e-05 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729786+0000", 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9327999999999997e-05 2024-09-19T21:35:04.170 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1456 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4554014379999991, 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028614296000000001, 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1456 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701341+0000", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701345+0000", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6969999999999999e-06 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701351+0000", 2024-09-19T21:35:04.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.122e-06 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701368+0000", 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7487999999999999e-05 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729825+0000", 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028456535000000002 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729885+0000", 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9738999999999998e-05 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729956+0000", 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1496999999999997e-05 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1457 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.175 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4553711140000001, 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028756075999999998, 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1457 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701374+0000", 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9319999999999999e-06 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701377+0000", 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.659e-06 2024-09-19T21:35:04.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701380+0000", 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8650000000000001e-06 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729994+0000", 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028613791 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730052+0000", 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8508999999999998e-05 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730128+0000", 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5683e-05 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.180 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1460 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4553609430000005, 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028920154999999999, 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1460 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701383+0000", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2389999999999999e-06 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701386+0000", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.717e-06 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701389+0000", 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8990000000000001e-06 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730165+0000", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02877569 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730227+0000", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2791000000000002e-05 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730302+0000", 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5036999999999993e-05 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.184 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1461 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4553488320000003, 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02910503, 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1461 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701397+0000", 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.576e-06 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701402+0000", 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4160000000000003e-06 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701407+0000", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730338+0000", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028931354999999999 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730404+0000", 2024-09-19T21:35:04.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5630999999999996e-05 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730499+0000", 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5397999999999997e-05 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:04.189 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552743249999995, 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.193 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552444980000008, 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.194 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552348120000005, 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.202 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552264860000008, 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:04.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:04.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552160940000007, 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.207 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:04.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4552055979999992, 2024-09-19T21:35:04.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1530 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4529496240000004, 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028447986000000001, 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1530 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703793+0000", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703807+0000", 2024-09-19T21:35:04.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4080000000000001e-05 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703816+0000", 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9670000000000007e-06 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703822+0000", 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732054+0000", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028232126999999999 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732131+0000", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7206000000000006e-05 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732242+0000", 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110623 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.219 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1531 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4529164249999997, 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028589459000000001, 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1531 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703826+0000", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703835+0000", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9910000000000008e-06 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703844+0000", 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9390000000000003e-06 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703849+0000", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.147e-06 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732283+0000", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028433429999999999 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732339+0000", 2024-09-19T21:35:04.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5804999999999998e-05 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732416+0000", 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7625000000000003e-05 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1534 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4528888179999999, 2024-09-19T21:35:04.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028735752, 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1534 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703856+0000", 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4669999999999998e-06 2024-09-19T21:35:04.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703866+0000", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2909999999999996e-06 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703879+0000", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3698e-05 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732452+0000", 2024-09-19T21:35:04.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028572441000000001 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732514+0000", 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1785999999999994e-05 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732590+0000", 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6440000000000007e-05 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1541 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4528592660000008, 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028868288999999998, 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1541 2024-09-19T21:35:04.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703890+0000", 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8199999999999999e-06 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703897+0000", 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703902+0000", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1170000000000003e-06 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732626+0000", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028723790999999999 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732683+0000", 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6645999999999999e-05 2024-09-19T21:35:04.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732752+0000", 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9426999999999998e-05 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1544 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4528122759999995, 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028982658000000001, 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1544 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703930+0000", 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703933+0000", 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7539999999999998e-06 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703939+0000", 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703945+0000", 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0159999999999999e-06 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732784+0000", 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028839773999999999 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732841+0000", 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6759000000000003e-05 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732914+0000", 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2564999999999994e-05 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1545 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4527908180000004, 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029130377999999998, 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1545 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703955+0000", 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6860000000000002e-06 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703968+0000", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3226e-05 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703974+0000", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9510000000000003e-06 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732951+0000", 2024-09-19T21:35:04.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028976842999999999 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733005+0000", 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4842000000000001e-05 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733083+0000", 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7380999999999997e-05 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4527615100000002, 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.243 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:04.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4275441240000006, 2024-09-19T21:35:04.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:04.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:04.253 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:04.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1396 write 0xac80a00 thru 0xac8d276 (0xc877 bytes) 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1397 punch from 0x78dc800 to 0x78e6c00, (0xa400 bytes) 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1398 write 0x8e07c00 thru 0x8e0ca38 (0x4e39 bytes) 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1401 read 0xabb6c00 thru 0xabc2731 (0xbb32 bytes) 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1403 write 0x611dc00 thru 0x6126630 (0x8a31 bytes) 2024-09-19T21:35:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:1404 write 0xe635c00 thru 0xe63afac (0x53ad bytes) 2024-09-19T21:35:04.257 INFO:teuthology.orchestra.run.smithi122.stdout:1405 read 0x1c26200 thru 0x1c33f4b (0xdd4c bytes) 2024-09-19T21:35:04.257 INFO:teuthology.orchestra.run.smithi122.stdout:1406 read 0x9175800 thru 0x918236b (0xcb6c bytes) 2024-09-19T21:35:04.257 INFO:teuthology.orchestra.run.smithi122.stdout:1407 read 0x9829800 thru 0x9836f5d (0xd75e bytes) 2024-09-19T21:35:04.257 INFO:teuthology.orchestra.run.smithi122.stdout:1410 write 0x771b600 thru 0x771f521 (0x3f22 bytes) 2024-09-19T21:35:04.257 INFO:teuthology.orchestra.run.smithi122.stdout:1411 trunc from 0xe63b000 to 0x3086600 2024-09-19T21:35:04.257 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:04.249+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:04.327 INFO:teuthology.orchestra.run.smithi122.stdout:1413 write 0xbd7a000 thru 0xbd86aa0 (0xcaa1 bytes) 2024-09-19T21:35:04.329 INFO:teuthology.orchestra.run.smithi122.stdout:1417 read 0x35e5000 thru 0x35f4f37 (0xff38 bytes) 2024-09-19T21:35:04.330 INFO:teuthology.orchestra.run.smithi122.stdout:1420 write 0x8b3200 thru 0x8bc3e1 (0x91e2 bytes) 2024-09-19T21:35:04.331 INFO:teuthology.orchestra.run.smithi122.stdout:1422 punch from 0x30f2200 to 0x30f8c00, (0x6a00 bytes) 2024-09-19T21:35:04.332 INFO:teuthology.orchestra.run.smithi122.stdout:1423 write 0xee9e00 thru 0xef8598 (0xe799 bytes) 2024-09-19T21:35:04.337 INFO:teuthology.orchestra.run.smithi122.stdout:1424 write 0x16de00 thru 0x1738c7 (0x5ac8 bytes) 2024-09-19T21:35:04.344 INFO:teuthology.orchestra.run.smithi122.stdout:1426 trunc from 0xbd86c00 to 0x89b7400 2024-09-19T21:35:04.355 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:04.349+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:04.372 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:04.372 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:04.372 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:04.373 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:04.375 INFO:teuthology.orchestra.run.smithi122.stdout:1428 read 0x1dca00 thru 0x1e08fb (0x3efc bytes) 2024-09-19T21:35:04.379 INFO:teuthology.orchestra.run.smithi122.stdout:1429 punch from 0x6d17400 to 0x6d1ee00, (0x7a00 bytes) 2024-09-19T21:35:04.380 INFO:teuthology.orchestra.run.smithi122.stdout:1434 write 0x143e400 thru 0x144d61b (0xf21c bytes) 2024-09-19T21:35:04.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:04.404 INFO:teuthology.orchestra.run.smithi122.stdout:1435 write 0x9f74c00 thru 0x9f7c391 (0x7792 bytes) 2024-09-19T21:35:04.406 INFO:teuthology.orchestra.run.smithi122.stdout:1437 read 0x872fe00 thru 0x873aa80 (0xac81 bytes) 2024-09-19T21:35:04.407 INFO:teuthology.orchestra.run.smithi122.stdout:1438 punch from 0x3708000 to 0x3708400, (0x400 bytes) 2024-09-19T21:35:04.408 INFO:teuthology.orchestra.run.smithi122.stdout:1440 punch from 0x6b3ee00 to 0x6b48e00, (0xa000 bytes) 2024-09-19T21:35:04.424 INFO:teuthology.orchestra.run.smithi122.stdout:1441 trunc from 0x9f7c400 to 0x3a4c200 2024-09-19T21:35:04.456 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:04.453+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:04.461 INFO:teuthology.orchestra.run.smithi122.stdout:1442 punch from 0x3ab600 to 0x3bae00, (0xf800 bytes) 2024-09-19T21:35:04.485 INFO:teuthology.orchestra.run.smithi122.stdout:1443 write 0x430e400 thru 0x43185f1 (0xa1f2 bytes) 2024-09-19T21:35:04.499 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:04.499 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:04.499 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:04.500 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:04.500 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:04.504 INFO:teuthology.orchestra.run.smithi122.stdout:1444 trunc from 0x4318600 to 0xb17e800 2024-09-19T21:35:04.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:04.527 INFO:teuthology.orchestra.run.smithi122.stdout:1447 write 0x52cb400 thru 0x52d2b84 (0x7785 bytes) 2024-09-19T21:35:04.530 INFO:teuthology.orchestra.run.smithi122.stdout:1449 write 0x2482600 thru 0x2491c57 (0xf658 bytes) 2024-09-19T21:35:04.536 INFO:teuthology.orchestra.run.smithi122.stdout:1450 trunc from 0xb17e800 to 0xcdca400 2024-09-19T21:35:04.548 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:04.548 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:04.549 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-19T21:35:04.549 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-19T21:35:04.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-19T21:35:04.557 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:04.553+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:04.561 INFO:teuthology.orchestra.run.smithi122.stdout:1451 write 0x4183200 thru 0x418bc55 (0x8a56 bytes) 2024-09-19T21:35:04.563 INFO:teuthology.orchestra.run.smithi122.stdout:1453 write 0x6958200 thru 0x6964c37 (0xca38 bytes) 2024-09-19T21:35:04.563 INFO:teuthology.orchestra.run.smithi122.stdout:1454 read 0x1d4fc00 thru 0x1d50775 (0xb76 bytes) 2024-09-19T21:35:04.564 INFO:teuthology.orchestra.run.smithi122.stdout:1456 read 0x1752e00 thru 0x175c8a3 (0x9aa4 bytes) 2024-09-19T21:35:04.565 INFO:teuthology.orchestra.run.smithi122.stdout:1458 read 0x8f1200 thru 0x8ff5d2 (0xe3d3 bytes) 2024-09-19T21:35:04.569 INFO:teuthology.orchestra.run.smithi122.stdout:1462 trunc from 0xcdca400 to 0xbeea000 2024-09-19T21:35:04.602 INFO:teuthology.orchestra.run.smithi122.stdout:1465 trunc from 0xbeea000 to 0xc68ba00 2024-09-19T21:35:04.625 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:04.625 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:04.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:04.625 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4296 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367711283999999, 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022203509999999999, 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4296 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253833+0000", 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.213e-06 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253840+0000", 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5930000000000002e-06 2024-09-19T21:35:04.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253845+0000", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3979999999999998e-06 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275901+0000", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022056172999999998 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275964+0000", 2024-09-19T21:35:04.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2723000000000002e-05 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276034+0000", 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9774e-05 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4297 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367691424, 2024-09-19T21:35:04.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359332999999999, 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4297 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:04.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253852+0000", 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4779999999999998e-06 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253858+0000", 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7740000000000001e-06 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253860+0000", 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276057+0000", 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022196344999999999 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276120+0000", 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3521000000000004e-05 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276209+0000", 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9135999999999995e-05 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4298 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367676874000001, 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022554268999999998, 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4298 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253864+0000", 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253873+0000", 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0289999999999994e-06 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253880+0000", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4729999999999997e-06 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253888+0000", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276235+0000", 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022346462000000001 2024-09-19T21:35:04.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276323+0000", 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7990000000000003e-05 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276419+0000", 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6291000000000004e-05 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4299 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 335872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367639275999998, 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022708860000000001, 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4299 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.640 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253905+0000", 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.591e-06 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253921+0000", 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5282e-05 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253927+0000", 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.003e-06 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276468+0000", 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022541017 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276535+0000", 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7119000000000006e-05 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276611+0000", 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6347000000000006e-05 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4300 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367611592999999, 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022876461000000001, 2024-09-19T21:35:04.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4300 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:04.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253932+0000", 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.857e-06 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253935+0000", 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253938+0000", 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.441e-06 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276636+0000", 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022698381 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276702+0000", 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5258000000000004e-05 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276806+0000", 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010488399999999999 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4301 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367601594, 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023114603000000001, 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4301 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253941+0000", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.44e-06 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253944+0000", 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0960000000000001e-06 2024-09-19T21:35:04.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253947+0000", 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4109999999999999e-06 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276839+0000", 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022892550000000001 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276947+0000", 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107793 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277055+0000", 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010752299999999999 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4302 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.657 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367367632000001, 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023063824, 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4302 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254173+0000", 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254179+0000", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0789999999999997e-06 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254190+0000", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0532e-05 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254196+0000", 2024-09-19T21:35:04.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6560000000000003e-06 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277079+0000", 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022882731 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277146+0000", 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6934000000000007e-05 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277238+0000", 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1891999999999994e-05 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4303 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367339057999999, 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023222402, 2024-09-19T21:35:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4303 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254205+0000", 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8329999999999998e-06 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254212+0000", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6549999999999998e-06 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254217+0000", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277263+0000", 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023045955 2024-09-19T21:35:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277333+0000", 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0257000000000002e-05 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277425+0000", 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1952000000000003e-05 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4304 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367295973000001, 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023378263, 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4304 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254245+0000", 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254248+0000", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2940000000000001e-06 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254256+0000", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254261+0000", 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5310000000000001e-06 2024-09-19T21:35:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277449+0000", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023187433 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277538+0000", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9474000000000004e-05 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277624+0000", 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5560000000000001e-05 2024-09-19T21:35:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4305 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 446464~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367271154000001, 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033521316000000002, 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4305 2024-09-19T21:35:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254273+0000", 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0850000000000001e-06 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254280+0000", 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0720000000000002e-06 2024-09-19T21:35:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254285+0000", 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3839999999999996e-06 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277648+0000", 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0233627 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277727+0000", 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9128999999999994e-05 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.287792+0000", 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010064559000000001 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4307 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.367090623999999, 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027454658999999999, 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4307 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254450+0000", 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254455+0000", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4560000000000002e-06 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254464+0000", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8570000000000007e-06 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254471+0000", 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:35:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281760+0000", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027288818999999999 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281834+0000", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4019999999999999e-05 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281906+0000", 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1353999999999995e-05 2024-09-19T21:35:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4309 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.366511774999999, 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027050399999999999, 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4309 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255029+0000", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255035+0000", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6500000000000001e-06 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255046+0000", 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1382999999999999e-05 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255061+0000", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.554e-05 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281929+0000", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02686796 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282019+0000", 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9860999999999999e-05 2024-09-19T21:35:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282080+0000", 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0891999999999999e-05 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4311 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.366329638, 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027091388000000001, 2024-09-19T21:35:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4311 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255211+0000", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255216+0000", 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3319999999999999e-06 2024-09-19T21:35:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255228+0000", 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1202999999999999e-05 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255235+0000", 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0350000000000004e-06 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282101+0000", 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026866476 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282202+0000", 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282303+0000", 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100883 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4313 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.366290663000001, 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027229189000000001, 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4313 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255250+0000", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255254+0000", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0629999999999999e-06 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255262+0000", 2024-09-19T21:35:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-06 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255269+0000", 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4050000000000003e-06 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282326+0000", 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027057184000000001 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282403+0000", 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6720999999999999e-05 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282480+0000", 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7001000000000004e-05 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4744 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352451974000001, 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022372296999999999, 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4744 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269089+0000", 2024-09-19T21:35:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269092+0000", 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269102+0000", 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0077e-05 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0289999999999996e-06 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291277+0000", 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022170091999999999 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291356+0000", 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9241999999999998e-05 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291462+0000", 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105609 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4746 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352430270999999, 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022538171999999999, 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4746 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269123+0000", 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0820000000000005e-06 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269128+0000", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0549999999999999e-06 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291488+0000", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359917 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291558+0000", 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0339000000000005e-05 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291649+0000", 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1558000000000007e-05 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4747 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352408623999999, 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022815518999999999, 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4747 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0185e-05 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7169999999999999e-06 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291712+0000", 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022561748 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291806+0000", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3826999999999995e-05 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291948+0000", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014276600000000001 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4749 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352387376999999, 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023058227000000001, 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4749 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269156+0000", 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2230000000000001e-06 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269165+0000", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8470000000000002e-06 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269170+0000", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8489999999999997e-06 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291977+0000", 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022806918999999998 2024-09-19T21:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1191000000000003e-05 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292212+0000", 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016450899999999999 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4751 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352367502, 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023246788000000001, 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4751 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6450000000000005e-06 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269191+0000", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9550000000000004e-06 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269196+0000", 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8500000000000002e-06 2024-09-19T21:35:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292239+0000", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023042450999999999 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292360+0000", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121804 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292421+0000", 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0467000000000002e-05 2024-09-19T21:35:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4753 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.352339646000001, 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023417911999999999, 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4753 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269204+0000", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.481e-06 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269213+0000", 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000005e-06 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269219+0000", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1919999999999996e-06 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292445+0000", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023226009999999998 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292517+0000", 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1898999999999997e-05 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292620+0000", 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103198 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:04.719 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:04.721 INFO:teuthology.orchestra.run.smithi122.stdout:1466 read 0xb268400 thru 0xb275928 (0xd529 bytes) 2024-09-19T21:35:04.721 INFO:teuthology.orchestra.run.smithi122.stdout:1468 trunc from 0xc68ba00 to 0x37a5400 2024-09-19T21:35:04.721 INFO:teuthology.orchestra.run.smithi122.stdout:1469 punch from 0xfc1e00 to 0xfcd000, (0xb200 bytes) 2024-09-19T21:35:04.721 INFO:teuthology.orchestra.run.smithi122.stdout:1471 read 0x3692a00 thru 0x369b572 (0x8b73 bytes) 2024-09-19T21:35:04.721 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:04.722 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:04.717+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:04.737 INFO:teuthology.orchestra.run.smithi122.stdout:1472 write 0xe7e6c00 thru 0xe7f294e (0xbd4f bytes) 2024-09-19T21:35:04.739 INFO:teuthology.orchestra.run.smithi122.stdout:1476 read 0xa3cc000 thru 0xa3d15c4 (0x55c5 bytes) 2024-09-19T21:35:04.739 INFO:teuthology.orchestra.run.smithi122.stdout:1479 write 0x4efc800 thru 0x4f055cb (0x8dcc bytes) 2024-09-19T21:35:04.740 INFO:teuthology.orchestra.run.smithi122.stdout:1481 read 0x3726400 thru 0x373596c (0xf56d bytes) 2024-09-19T21:35:04.741 INFO:teuthology.orchestra.run.smithi122.stdout:1483 read 0x29ba200 thru 0x29c2fa9 (0x8daa bytes) 2024-09-19T21:35:04.742 INFO:teuthology.orchestra.run.smithi122.stdout:1487 punch from 0x4ff8e00 to 0x5006a00, (0xdc00 bytes) 2024-09-19T21:35:04.743 INFO:teuthology.orchestra.run.smithi122.stdout:1488 read 0x6c33800 thru 0x6c358c3 (0x20c4 bytes) 2024-09-19T21:35:04.744 INFO:teuthology.orchestra.run.smithi122.stdout:1489 punch from 0xa473e00 to 0xa47ea00, (0xac00 bytes) 2024-09-19T21:35:04.744 INFO:teuthology.orchestra.run.smithi122.stdout:1494 write 0x1315200 thru 0x131b590 (0x6391 bytes) 2024-09-19T21:35:04.747 INFO:teuthology.orchestra.run.smithi122.stdout:1495 read 0x60c6000 thru 0x60cdab9 (0x7aba bytes) 2024-09-19T21:35:04.749 INFO:teuthology.orchestra.run.smithi122.stdout:1496 trunc from 0xe7f2a00 to 0xdfac200 2024-09-19T21:35:04.774 INFO:teuthology.orchestra.run.smithi122.stdout:1499 read 0xb5d9400 thru 0xb5df769 (0x636a bytes) 2024-09-19T21:35:04.775 INFO:teuthology.orchestra.run.smithi122.stdout:1500 read 0x9084c00 thru 0x909488a (0xfc8b bytes) 2024-09-19T21:35:04.777 INFO:teuthology.orchestra.run.smithi122.stdout:1501 write 0x8efc00 thru 0x8f6fc5 (0x73c6 bytes) 2024-09-19T21:35:04.778 INFO:teuthology.orchestra.run.smithi122.stdout:1503 read 0xaaa600 thru 0xab223c (0x7c3d bytes) 2024-09-19T21:35:04.779 INFO:teuthology.orchestra.run.smithi122.stdout:1504 write 0x50bc800 thru 0x50c52e9 (0x8aea bytes) 2024-09-19T21:35:04.780 INFO:teuthology.orchestra.run.smithi122.stdout:1505 read 0x2ee7800 thru 0x2ef6a7a (0xf27b bytes) 2024-09-19T21:35:04.781 INFO:teuthology.orchestra.run.smithi122.stdout:1508 read 0x5bd000 thru 0x5cc4c2 (0xf4c3 bytes) 2024-09-19T21:35:04.783 INFO:teuthology.orchestra.run.smithi122.stdout:1509 read 0xd0baa00 thru 0xd0bd99e (0x2f9f bytes) 2024-09-19T21:35:04.783 INFO:teuthology.orchestra.run.smithi122.stdout:1510 write 0x2e78400 thru 0x2e7d7d8 (0x53d9 bytes) 2024-09-19T21:35:04.794 INFO:teuthology.orchestra.run.smithi122.stdout:1515 trunc from 0xdfac200 to 0xa058c00 2024-09-19T21:35:04.821 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:04.817+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:04.835 INFO:teuthology.orchestra.run.smithi122.stdout:1522 read 0x2996000 thru 0x29a0b14 (0xab15 bytes) 2024-09-19T21:35:04.838 INFO:teuthology.orchestra.run.smithi122.stdout:1523 read 0x1117200 thru 0x1125996 (0xe797 bytes) 2024-09-19T21:35:04.840 INFO:teuthology.orchestra.run.smithi122.stdout:1524 read 0x8f5b400 thru 0x8f60086 (0x4c87 bytes) 2024-09-19T21:35:04.841 INFO:teuthology.orchestra.run.smithi122.stdout:1525 write 0x3be3800 thru 0x3bef2ed (0xbaee bytes) 2024-09-19T21:35:04.841 INFO:teuthology.orchestra.run.smithi122.stdout:1526 read 0x7bb6200 thru 0x7bc39e0 (0xd7e1 bytes) 2024-09-19T21:35:04.870 INFO:teuthology.orchestra.run.smithi122.stdout:1527 write 0xc11f000 thru 0xc1280ea (0x90eb bytes) 2024-09-19T21:35:04.871 INFO:teuthology.orchestra.run.smithi122.stdout:1533 punch from 0x52bea00 to 0x52cc600, (0xdc00 bytes) 2024-09-19T21:35:04.872 INFO:teuthology.orchestra.run.smithi122.stdout:1534 write 0x6372200 thru 0x637a3a7 (0x81a8 bytes) 2024-09-19T21:35:04.873 INFO:teuthology.orchestra.run.smithi122.stdout:1536 trunc from 0xc128200 to 0xc36c000 2024-09-19T21:35:04.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:04.888 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:04.889 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:04.889 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:04.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:04.903 INFO:teuthology.orchestra.run.smithi122.stdout:1537 read 0x7fada00 thru 0x7fb28bd (0x4ebe bytes) 2024-09-19T21:35:04.905 INFO:teuthology.orchestra.run.smithi122.stdout:1538 write 0x9b96600 thru 0x9b9aff2 (0x49f3 bytes) 2024-09-19T21:35:04.906 INFO:teuthology.orchestra.run.smithi122.stdout:1539 read 0xb0ba400 thru 0xb0bd376 (0x2f77 bytes) 2024-09-19T21:35:04.906 INFO:teuthology.orchestra.run.smithi122.stdout:1543 write 0x311e200 thru 0x312417e (0x5f7f bytes) 2024-09-19T21:35:04.925 INFO:teuthology.orchestra.run.smithi122.stdout:1546 trunc from 0xc36c000 to 0x488a600 2024-09-19T21:35:04.944 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:04.941+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:04.975 INFO:teuthology.orchestra.run.smithi122.stdout:1547 punch from 0x8d6e00 to 0x8e5a00, (0xec00 bytes) 2024-09-19T21:35:04.977 INFO:teuthology.orchestra.run.smithi122.stdout:1549 read 0xc9b200 thru 0xca649d (0xb29e bytes) 2024-09-19T21:35:04.983 INFO:teuthology.orchestra.run.smithi122.stdout:1551 trunc from 0x488a600 to 0x2e27800 2024-09-19T21:35:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:05.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:05.023 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:05.017+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:05.044 INFO:teuthology.orchestra.run.smithi122.stdout:1552 write 0x65a8c00 thru 0x65ae440 (0x5841 bytes) 2024-09-19T21:35:05.046 INFO:teuthology.orchestra.run.smithi122.stdout:1553 read 0x2889400 thru 0x288eb11 (0x5712 bytes) 2024-09-19T21:35:05.049 INFO:teuthology.orchestra.run.smithi122.stdout:1555 punch from 0x2e8c400 to 0x2e91c00, (0x5800 bytes) 2024-09-19T21:35:05.050 INFO:teuthology.orchestra.run.smithi122.stdout:1556 read 0x512b400 thru 0x513571c (0xa31d bytes) 2024-09-19T21:35:05.057 INFO:teuthology.orchestra.run.smithi122.stdout:1557 read 0x424e00 thru 0x424e22 (0x23 bytes) 2024-09-19T21:35:05.058 INFO:teuthology.orchestra.run.smithi122.stdout:1558 punch from 0x3c4e800 to 0x3c56a00, (0x8200 bytes) 2024-09-19T21:35:05.095 INFO:teuthology.orchestra.run.smithi122.stdout:1560 write 0xe010a00 thru 0xe0111fd (0x7fe bytes) 2024-09-19T21:35:05.098 INFO:teuthology.orchestra.run.smithi122.stdout:1562 read 0xbb1da00 thru 0xbb2b3a4 (0xd9a5 bytes) 2024-09-19T21:35:05.099 INFO:teuthology.orchestra.run.smithi122.stdout:1564 read 0x3a1b800 thru 0x3a1e874 (0x3075 bytes) 2024-09-19T21:35:05.100 INFO:teuthology.orchestra.run.smithi122.stdout:1567 punch from 0x166ba00 to 0x1678400, (0xca00 bytes) 2024-09-19T21:35:05.101 INFO:teuthology.orchestra.run.smithi122.stdout:1568 punch from 0xc33f200 to 0xc341e00, (0x2c00 bytes) 2024-09-19T21:35:05.101 INFO:teuthology.orchestra.run.smithi122.stdout:1569 write 0x2f5e000 thru 0x2f69c76 (0xbc77 bytes) 2024-09-19T21:35:05.104 INFO:teuthology.orchestra.run.smithi122.stdout:1572 read 0x55ccc00 thru 0x55dc803 (0xfc04 bytes) 2024-09-19T21:35:05.124 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:05.117+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:05.127 INFO:teuthology.orchestra.run.smithi122.stdout:1574 trunc from 0xe011200 to 0x2584c00 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.734436469999999, 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:35:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:35:05.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:05.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.734416162999999, 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:05.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:35:05.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.734396701000001, 2024-09-19T21:35:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:35:05.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:35:05.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.734362390000001, 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:35:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.734336890000002, 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:35:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730383661000001, 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:35:05.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:35:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:35:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730346536999999, 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:35:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:35:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730309318, 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:35:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730264929000001, 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:35:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730247242000001, 2024-09-19T21:35:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:35:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730222292000001, 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:35:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730183001, 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730145598, 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:35:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.730127768999999, 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:35:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726478991, 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:35:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726454446999998, 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726435832, 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:35:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726386426000001, 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:35:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:35:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726368028, 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:35:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.726342258999999, 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:35:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.235 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:05.237 INFO:teuthology.orchestra.run.smithi122.stdout:1577 read 0x19b9400 thru 0x19bc4d2 (0x30d3 bytes) 2024-09-19T21:35:05.237 INFO:teuthology.orchestra.run.smithi122.stdout:1578 read 0x1344800 thru 0x1351390 (0xcb91 bytes) 2024-09-19T21:35:05.237 INFO:teuthology.orchestra.run.smithi122.stdout:1579 trunc from 0x2584c00 to 0xdce1a00 2024-09-19T21:35:05.237 INFO:teuthology.orchestra.run.smithi122.stdout:1580 punch from 0x68a3c00 to 0x68af400, (0xb800 bytes) 2024-09-19T21:35:05.237 INFO:teuthology.orchestra.run.smithi122.stdout:1585 punch from 0xc171200 to 0xc172400, (0x1200 bytes) 2024-09-19T21:35:05.238 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:05.238 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:05.233+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:05.254 INFO:teuthology.orchestra.run.smithi122.stdout:1587 write 0xea5d400 thru 0xea63567 (0x6168 bytes) 2024-09-19T21:35:05.257 INFO:teuthology.orchestra.run.smithi122.stdout:1592 write 0x3ad2a00 thru 0x3ad9d6b (0x736c bytes) 2024-09-19T21:35:05.296 INFO:teuthology.orchestra.run.smithi122.stdout:1593 trunc from 0xea63600 to 0x1096e00 2024-09-19T21:35:05.319 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-19T21:35:05.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:35:05.376 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:05.373+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:05.400 INFO:teuthology.orchestra.run.smithi122.stdout:1595 write 0xec33a00 thru 0xec38c10 (0x5211 bytes) 2024-09-19T21:35:05.401 INFO:teuthology.orchestra.run.smithi122.stdout:1596 punch from 0xd441000 to 0xd441c00, (0xc00 bytes) 2024-09-19T21:35:05.402 INFO:teuthology.orchestra.run.smithi122.stdout:1598 read 0xb19ca00 thru 0xb1a7dc0 (0xb3c1 bytes) 2024-09-19T21:35:05.402 INFO:teuthology.orchestra.run.smithi122.stdout:1599 punch from 0xba50200 to 0xba5a800, (0xa600 bytes) 2024-09-19T21:35:05.404 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:05.405 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:05.405 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:05.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:05.424 INFO:teuthology.orchestra.run.smithi122.stdout:1600 trunc from 0xec38e00 to 0x2753e00 2024-09-19T21:35:05.438 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:05.433+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:05.460 INFO:teuthology.orchestra.run.smithi122.stdout:1602 trunc from 0x2753e00 to 0x240fc00 2024-09-19T21:35:05.486 INFO:teuthology.orchestra.run.smithi122.stdout:1603 punch from 0x38d000 to 0x396800, (0x9800 bytes) 2024-09-19T21:35:05.487 INFO:teuthology.orchestra.run.smithi122.stdout:1604 read 0x12f9200 thru 0x12fa3e5 (0x11e6 bytes) 2024-09-19T21:35:05.490 INFO:teuthology.orchestra.run.smithi122.stdout:1606 read 0x1fb8a00 thru 0x1fbfdb4 (0x73b5 bytes) 2024-09-19T21:35:05.491 INFO:teuthology.orchestra.run.smithi122.stdout:1608 read 0xb6f000 thru 0xb70437 (0x1438 bytes) 2024-09-19T21:35:05.517 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:05.517 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:05.517 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:05.517 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:05.518 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:05.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:05.530 INFO:teuthology.orchestra.run.smithi122.stdout:1609 write 0x9bd5000 thru 0x9bd6dd6 (0x1dd7 bytes) 2024-09-19T21:35:05.532 INFO:teuthology.orchestra.run.smithi122.stdout:1610 read 0x5446a00 thru 0x5449341 (0x2942 bytes) 2024-09-19T21:35:05.535 INFO:teuthology.orchestra.run.smithi122.stdout:1612 write 0x7476a00 thru 0x7478611 (0x1c12 bytes) 2024-09-19T21:35:05.539 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:05.533+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:05.541 INFO:teuthology.orchestra.run.smithi122.stdout:1615 trunc from 0x9bd6e00 to 0xbafa800 2024-09-19T21:35:05.570 INFO:teuthology.orchestra.run.smithi122.stdout:1616 write 0x18ce600 thru 0x18d948c (0xae8d bytes) 2024-09-19T21:35:05.572 INFO:teuthology.orchestra.run.smithi122.stdout:1617 trunc from 0xbafa800 to 0xbcd0600 2024-09-19T21:35:05.593 INFO:teuthology.orchestra.run.smithi122.stdout:1618 write 0x669de00 thru 0x66ab1c7 (0xd3c8 bytes) 2024-09-19T21:35:05.595 INFO:teuthology.orchestra.run.smithi122.stdout:1620 punch from 0x52b2200 to 0x52bc200, (0xa000 bytes) 2024-09-19T21:35:05.596 INFO:teuthology.orchestra.run.smithi122.stdout:1625 read 0x25c8a00 thru 0x25d829d (0xf89e bytes) 2024-09-19T21:35:05.601 INFO:teuthology.orchestra.run.smithi122.stdout:1626 write 0x6579e00 thru 0x657e585 (0x4786 bytes) 2024-09-19T21:35:05.628 INFO:teuthology.orchestra.run.smithi122.stdout:1628 write 0xd99c400 thru 0xd9a03aa (0x3fab bytes) 2024-09-19T21:35:05.629 INFO:teuthology.orchestra.run.smithi122.stdout:1631 write 0x23b2e00 thru 0x23ba2da (0x74db bytes) 2024-09-19T21:35:05.630 INFO:teuthology.orchestra.run.smithi122.stdout:1633 write 0x2bd9800 thru 0x2be19b4 (0x81b5 bytes) 2024-09-19T21:35:05.633 INFO:teuthology.orchestra.run.smithi122.stdout:1634 read 0x7cf2800 thru 0x7cf354b (0xd4c bytes) 2024-09-19T21:35:05.633 INFO:teuthology.orchestra.run.smithi122.stdout:1635 write 0x4711400 thru 0x471f870 (0xe471 bytes) 2024-09-19T21:35:05.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:05.634 INFO:teuthology.orchestra.run.smithi122.stdout:1639 read 0x2eb000 thru 0x2f79cf (0xc9d0 bytes) 2024-09-19T21:35:05.635 INFO:teuthology.orchestra.run.smithi122.stdout:1640 read 0xa442400 thru 0xa449a60 (0x7661 bytes) 2024-09-19T21:35:05.636 INFO:teuthology.orchestra.run.smithi122.stdout:1641 read 0x7e52000 thru 0x7e53751 (0x1752 bytes) 2024-09-19T21:35:05.636 INFO:teuthology.orchestra.run.smithi122.stdout:1642 read 0x1f33e00 thru 0x1f414bf (0xd6c0 bytes) 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1075 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 7168~512] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9249451999999998, 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030194739000000002, 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1075 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712126+0000", 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5739999999999998e-06 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712134+0000", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6550000000000004e-06 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712139+0000", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2270000000000001e-06 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742210+0000", 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030070403999999998 2024-09-19T21:35:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742253+0000", 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3010000000000003e-05 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742319+0000", 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6241000000000005e-05 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1077 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9249245130000006, 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030330979000000001, 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1077 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712144+0000", 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712148+0000", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3330000000000001e-06 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712156+0000", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3280000000000006e-06 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712161+0000", 2024-09-19T21:35:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2299999999999999e-06 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742353+0000", 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030191913000000001 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742398+0000", 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4669000000000002e-05 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742476+0000", 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8181000000000002e-05 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1480 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9048693080000003, 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030120475000000001, 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1480 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732201+0000", 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4160000000000001e-06 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732205+0000", 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2169999999999996e-06 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732208+0000", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6259999999999999e-06 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762084+0000", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029876117000000001 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762193+0000", 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108906 2024-09-19T21:35:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762321+0000", 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012741400000000001 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1485 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9048589969999998, 2024-09-19T21:35:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030359516999999999, 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1485 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732212+0000", 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5269999999999999e-06 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732215+0000", 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.692e-06 2024-09-19T21:35:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732218+0000", 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6010000000000002e-06 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762393+0000", 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030175415000000001 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762470+0000", 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6774999999999996e-05 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762570+0000", 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9883000000000006e-05 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1540 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9008076569999997, 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.03012836, 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1540 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736261+0000", 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736264+0000", 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3120000000000001e-06 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736269+0000", 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736274+0000", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.69e-06 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766159+0000", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029884984 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766269+0000", 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110098 2024-09-19T21:35:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766390+0000", 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121489 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1543 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007906590000001, 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030340459, 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1543 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736278+0000", 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736281+0000", 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4250000000000001e-06 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736286+0000", 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.665e-06 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736293+0000", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5039999999999999e-06 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766437+0000", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030144388000000001 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766523+0000", 2024-09-19T21:35:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5575000000000007e-05 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766619+0000", 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6299999999999996e-05 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1546 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007704099999998, 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030580726999999999, 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1546 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.672 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:05.633+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736300+0000", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8369999999999999e-06 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736304+0000", 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9890000000000003e-06 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736314+0000", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2550000000000003e-06 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766650+0000", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030336609 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766753+0000", 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102413 2024-09-19T21:35:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766880+0000", 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012705999999999999 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1551 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007528570000005, 2024-09-19T21:35:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030830699999999999, 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1551 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736316+0000", 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736322+0000", 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8279999999999999e-06 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736325+0000", 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.348e-06 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736328+0000", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7020000000000001e-06 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766911+0000", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030582668 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767012+0000", 2024-09-19T21:35:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101053 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767147+0000", 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135597 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1554 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007392680000006, 2024-09-19T21:35:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031084074, 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1554 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736331+0000", 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5719999999999999e-06 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736334+0000", 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9050000000000001e-06 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736337+0000", 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6429999999999999e-06 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767184+0000", 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030846625999999999 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767286+0000", 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010242399999999999 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767414+0000", 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012813500000000001 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1559 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9007089999999991, 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031279296999999998, 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1559 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736362+0000", 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.362e-06 2024-09-19T21:35:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736364+0000", 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.695e-06 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736367+0000", 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767459+0000", 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031091534000000001 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767540+0000", 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1353000000000005e-05 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767640+0000", 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9806e-05 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1562 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9006990619999993, 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031533528999999998, 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1562 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736371+0000", 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.373e-06 2024-09-19T21:35:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736374+0000", 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8480000000000001e-06 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736378+0000", 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8850000000000001e-06 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767680+0000", 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031301571 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767772+0000", 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2255000000000004e-05 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767904+0000", 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181299999999999 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1569 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9006586609999996, 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031758058999999998, 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1569 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736410+0000", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736413+0000", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6960000000000001e-06 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736421+0000", 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6399999999999997e-06 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736425+0000", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5279999999999997e-06 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767935+0000", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031509713000000002 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768041+0000", 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106397 2024-09-19T21:35:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768169+0000", 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000127479 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1571 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9006128980000003, 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031967883000000002, 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1571 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736459+0000", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736466+0000", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5250000000000002e-06 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736470+0000", 2024-09-19T21:35:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768196+0000", 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031726035 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768298+0000", 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010195400000000001 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768424+0000", 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012614899999999999 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1572 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9005945339999997, 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032212024999999998, 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1572 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736474+0000", 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736477+0000", 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4940000000000002e-06 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736482+0000", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5010000000000004e-06 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736488+0000", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1079999999999998e-06 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768452+0000", 2024-09-19T21:35:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031963671999999999 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768557+0000", 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104411 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768687+0000", 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000130307 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1574 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9005767910000007, 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032396917999999997, 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1574 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736492+0000", 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736495+0000", 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3640000000000002e-06 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736500+0000", 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7069999999999997e-06 2024-09-19T21:35:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736505+0000", 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5959999999999997e-06 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768731+0000", 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032225666 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768798+0000", 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7187000000000006e-05 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768890+0000", 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1730000000000004e-05 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1575 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9005594450000007, 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032552508000000001, 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1575 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736512+0000", 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736518+0000", 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736523+0000", 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5670000000000002e-06 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768914+0000", 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032390713000000002 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768976+0000", 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2390000000000004e-05 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769062+0000", 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6248000000000005e-05 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1578 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9005077450000005, 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032774279000000003, 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1578 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736561+0000", 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736565+0000", 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.557e-06 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736572+0000", 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9419999999999998e-06 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736576+0000", 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4730000000000002e-06 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769102+0000", 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032525674999999997 2024-09-19T21:35:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769205+0000", 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010330700000000001 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769336+0000", 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013078699999999999 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1579 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9004885599999994, 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033004987999999999, 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1579 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736580+0000", 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736583+0000", 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8090000000000001e-06 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736589+0000", 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-19T21:35:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736593+0000", 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5210000000000004e-06 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769367+0000", 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032773434999999997 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769463+0000", 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6069000000000005e-05 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769586+0000", 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012306200000000001 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1582 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.9004075040000004, 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033168969999999999, 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1582 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736661+0000", 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736663+0000", 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.872e-06 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736668+0000", 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9280000000000001e-06 2024-09-19T21:35:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736677+0000", 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2709999999999992e-06 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769651+0000", 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032974366999999997 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769726+0000", 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5244e-05 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769831+0000", 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104737 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8884475720000005, 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:05.741 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:05.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:05.743 INFO:teuthology.orchestra.run.smithi122.stdout:1643 trunc from 0xd9a0400 to 0x2a89600 2024-09-19T21:35:05.743 INFO:teuthology.orchestra.run.smithi122.stdout:1644 punch from 0x956400 to 0x956800, (0x400 bytes) 2024-09-19T21:35:05.743 INFO:teuthology.orchestra.run.smithi122.stdout:1646 write 0x2d0ae00 thru 0x2d10380 (0x5581 bytes) 2024-09-19T21:35:05.743 INFO:teuthology.orchestra.run.smithi122.stdout:1647 trunc from 0x2d10400 to 0xf15c00 2024-09-19T21:35:05.744 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:05.737+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:05.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:05.776 INFO:teuthology.orchestra.run.smithi122.stdout:1648 trunc from 0xf15c00 to 0x3103a00 2024-09-19T21:35:05.787 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:05.798 INFO:teuthology.orchestra.run.smithi122.stdout:1654 write 0x2103a00 thru 0x21099ab (0x5fac bytes) 2024-09-19T21:35:05.802 INFO:teuthology.orchestra.run.smithi122.stdout:1655 write 0x16f3000 thru 0x16f9995 (0x6996 bytes) 2024-09-19T21:35:05.806 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:05.830 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:05.844 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:05.841+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:05.850 INFO:teuthology.orchestra.run.smithi122.stdout:1656 write 0x902ae00 thru 0x903a1c8 (0xf3c9 bytes) 2024-09-19T21:35:05.857 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:05.857 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:05.858 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:05.863 INFO:teuthology.orchestra.run.smithi122.stdout:1657 trunc from 0x903a200 to 0xda3f600 2024-09-19T21:35:05.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:05.881 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:05.897 INFO:teuthology.orchestra.run.smithi122.stdout:1658 read 0x4292600 thru 0x429950d (0x6f0e bytes) 2024-09-19T21:35:05.899 INFO:teuthology.orchestra.run.smithi122.stdout:1659 read 0x3bd1e00 thru 0x3bded4d (0xcf4e bytes) 2024-09-19T21:35:05.901 INFO:teuthology.orchestra.run.smithi122.stdout:1660 read 0x56d2c00 thru 0x56dd2d2 (0xa6d3 bytes) 2024-09-19T21:35:05.902 INFO:teuthology.orchestra.run.smithi122.stdout:1667 read 0x387c600 thru 0x3886405 (0x9e06 bytes) 2024-09-19T21:35:05.903 INFO:teuthology.orchestra.run.smithi122.stdout:1670 read 0xa58c400 thru 0xa594137 (0x7d38 bytes) 2024-09-19T21:35:05.915 INFO:teuthology.orchestra.run.smithi122.stdout:1671 trunc from 0xda3f600 to 0x8c86800 2024-09-19T21:35:05.939 INFO:teuthology.orchestra.run.smithi122.stdout:1672 punch from 0x7b2ce00 to 0x7b3a200, (0xd400 bytes) 2024-09-19T21:35:05.940 INFO:teuthology.orchestra.run.smithi122.stdout:1673 read 0x791be00 thru 0x79255ab (0x97ac bytes) 2024-09-19T21:35:05.948 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:05.945+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:05.949 INFO:teuthology.orchestra.run.smithi122.stdout:1674 trunc from 0x8c86800 to 0xcca4400 2024-09-19T21:35:05.967 INFO:teuthology.orchestra.run.smithi122.stdout:1677 write 0x67cf600 thru 0x67d211f (0x2b20 bytes) 2024-09-19T21:35:05.968 INFO:teuthology.orchestra.run.smithi122.stdout:1679 read 0x5c2f800 thru 0x5c38c7d (0x947e bytes) 2024-09-19T21:35:05.970 INFO:teuthology.orchestra.run.smithi122.stdout:1680 write 0xa78ea00 thru 0xa79e2ec (0xf8ed bytes) 2024-09-19T21:35:05.970 INFO:teuthology.orchestra.run.smithi122.stdout:1681 read 0x7623a00 thru 0x76329be (0xefbf bytes) 2024-09-19T21:35:05.970 INFO:teuthology.orchestra.run.smithi122.stdout:1682 read 0xb2ac200 thru 0xb2bbab0 (0xf8b1 bytes) 2024-09-19T21:35:05.971 INFO:teuthology.orchestra.run.smithi122.stdout:1683 read 0x40bdc00 thru 0x40c2add (0x4ede bytes) 2024-09-19T21:35:05.971 INFO:teuthology.orchestra.run.smithi122.stdout:1688 read 0x1df400 thru 0x1e752f (0x8130 bytes) 2024-09-19T21:35:05.973 INFO:teuthology.orchestra.run.smithi122.stdout:1689 read 0x5a84e00 thru 0x5a8bf92 (0x7193 bytes) 2024-09-19T21:35:05.974 INFO:teuthology.orchestra.run.smithi122.stdout:1691 write 0x8e12200 thru 0x8e13d30 (0x1b31 bytes) 2024-09-19T21:35:05.974 INFO:teuthology.orchestra.run.smithi122.stdout:1692 read 0x4f3b800 thru 0x4f4a05a (0xe85b bytes) 2024-09-19T21:35:05.976 INFO:teuthology.orchestra.run.smithi122.stdout:1696 read 0xb4bc000 thru 0xb4c901d (0xd01e bytes) 2024-09-19T21:35:05.976 INFO:teuthology.orchestra.run.smithi122.stdout:1697 punch from 0x3dfc600 to 0x3e0c000, (0xfa00 bytes) 2024-09-19T21:35:05.977 INFO:teuthology.orchestra.run.smithi122.stdout:1702 punch from 0xbb1f200 to 0xbb29a00, (0xa800 bytes) 2024-09-19T21:35:05.978 INFO:teuthology.orchestra.run.smithi122.stdout:1704 write 0x6722e00 thru 0x67319ff (0xec00 bytes) 2024-09-19T21:35:05.996 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:05.997 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:05.997 INFO:teuthology.orchestra.run.smithi122.stdout:1705 trunc from 0xcca4400 to 0x282c400 2024-09-19T21:35:06.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:06.046 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:06.041+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:06.080 INFO:teuthology.orchestra.run.smithi122.stdout:1706 write 0x7dde400 thru 0x7debe40 (0xda41 bytes) 2024-09-19T21:35:06.082 INFO:teuthology.orchestra.run.smithi122.stdout:1708 punch from 0x2eeb400 to 0x2ef3600, (0x8200 bytes) 2024-09-19T21:35:06.083 INFO:teuthology.orchestra.run.smithi122.stdout:1714 read 0x48b9e00 thru 0x48c5bbd (0xbdbe bytes) 2024-09-19T21:35:06.083 INFO:teuthology.orchestra.run.smithi122.stdout:1715 read 0x54c4c00 thru 0x54cf05e (0xa45f bytes) 2024-09-19T21:35:06.085 INFO:teuthology.orchestra.run.smithi122.stdout:1716 punch from 0x2cf8000 to 0x2cfb800, (0x3800 bytes) 2024-09-19T21:35:06.085 INFO:teuthology.orchestra.run.smithi122.stdout:1717 write 0x3a1a200 thru 0x3a1d0b2 (0x2eb3 bytes) 2024-09-19T21:35:06.115 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.115 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:06.129 INFO:teuthology.orchestra.run.smithi122.stdout:1718 write 0xc6d3000 thru 0xc6da651 (0x7652 bytes) 2024-09-19T21:35:06.132 INFO:teuthology.orchestra.run.smithi122.stdout:1719 punch from 0x3fcca00 to 0x3fdae00, (0xe400 bytes) 2024-09-19T21:35:06.133 INFO:teuthology.orchestra.run.smithi122.stdout:1726 read 0x392000 thru 0x395bbb (0x3bbc bytes) 2024-09-19T21:35:06.139 INFO:teuthology.orchestra.run.smithi122.stdout:1727 write 0x99c1c00 thru 0x99c8adf (0x6ee0 bytes) 2024-09-19T21:35:06.145 INFO:teuthology.orchestra.run.smithi122.stdout:1728 trunc from 0xc6da800 to 0xa9d1800 2024-09-19T21:35:06.168 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:06.165+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:06.198 INFO:teuthology.orchestra.run.smithi122.stdout:1732 punch from 0x96f4000 to 0x96f9600, (0x5600 bytes) 2024-09-19T21:35:06.199 INFO:teuthology.orchestra.run.smithi122.stdout:1733 read 0x639800 thru 0x63b1a3 (0x19a4 bytes) 2024-09-19T21:35:06.202 INFO:teuthology.orchestra.run.smithi122.stdout:1736 write 0x4ac7000 thru 0x4ad41d5 (0xd1d6 bytes) 2024-09-19T21:35:06.203 INFO:teuthology.orchestra.run.smithi122.stdout:1741 read 0x8f7ec00 thru 0x8f85619 (0x6a1a bytes) 2024-09-19T21:35:06.204 INFO:teuthology.orchestra.run.smithi122.stdout:1742 read 0x791a00 thru 0x793546 (0x1b47 bytes) 2024-09-19T21:35:06.205 INFO:teuthology.orchestra.run.smithi122.stdout:1743 write 0x6761c00 thru 0x67716fe (0xfaff bytes) 2024-09-19T21:35:06.206 INFO:teuthology.orchestra.run.smithi122.stdout:1747 write 0x5ac9c00 thru 0x5acb1b3 (0x15b4 bytes) 2024-09-19T21:35:06.207 INFO:teuthology.orchestra.run.smithi122.stdout:1748 clone 13 order 23 su 131072 sc 4 2024-09-19T21:35:06.247 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:06.241+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:06.284 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.284 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:06.332 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:35:06.348 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:06.345+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:06.410 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.410 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:06.458 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone12 from 0xa9d1800 (overlap 0xf15c00) to 0x182200 2024-09-19T21:35:06.464 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:06.461+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:06.550 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:06.545+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:06.577 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.577 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.587 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:06.651 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:06.645+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:06.706 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.706 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:06.751 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:06.749+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:06.853 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:06.849+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:06.875 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.875 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:06.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:06.954 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:06.949+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:06.994 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:06.994 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:07.055 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:07.049+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:07.087 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-19T21:35:07.126 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.126 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:07.156 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:07.153+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:07.257 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:07.253+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:07.263 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.263 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:07.358 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:07.353+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:07.405 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.405 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:07.460 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:07.457+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:07.560 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:07.558+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:07.570 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.571 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:07.662 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:07.658+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:07.679 INFO:teuthology.orchestra.run.smithi122.stdout:1750 trunc from 0xa9d1800 to 0x9721600 2024-09-19T21:35:07.685 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.685 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:07.755 INFO:teuthology.orchestra.run.smithi122.stdout:1753 trunc from 0x9721600 to 0xb473000 2024-09-19T21:35:07.762 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:07.757+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:07.777 INFO:teuthology.orchestra.run.smithi122.stdout:1756 read 0x2f88400 thru 0x2f8982a (0x142b bytes) 2024-09-19T21:35:07.779 INFO:teuthology.orchestra.run.smithi122.stdout:1757 write 0x81db000 thru 0x81e0ffa (0x5ffb bytes) 2024-09-19T21:35:07.780 INFO:teuthology.orchestra.run.smithi122.stdout:1758 punch from 0x39fc200 to 0x3a02000, (0x5e00 bytes) 2024-09-19T21:35:07.783 INFO:teuthology.orchestra.run.smithi122.stdout:1759 punch from 0x550f000 to 0x5519c00, (0xac00 bytes) 2024-09-19T21:35:07.785 INFO:teuthology.orchestra.run.smithi122.stdout:1760 punch from 0x83a8a00 to 0x83adc00, (0x5200 bytes) 2024-09-19T21:35:07.810 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.810 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:07.822 INFO:teuthology.orchestra.run.smithi122.stdout:1763 write 0xdbc0a00 thru 0xdbd00d6 (0xf6d7 bytes) 2024-09-19T21:35:07.824 INFO:teuthology.orchestra.run.smithi122.stdout:1764 write 0x7dc2400 thru 0x7dcef23 (0xcb24 bytes) 2024-09-19T21:35:07.825 INFO:teuthology.orchestra.run.smithi122.stdout:1765 read 0xa59f000 thru 0xa5ad85e (0xe85f bytes) 2024-09-19T21:35:07.825 INFO:teuthology.orchestra.run.smithi122.stdout:1767 read 0x4931000 thru 0x493f447 (0xe448 bytes) 2024-09-19T21:35:07.827 INFO:teuthology.orchestra.run.smithi122.stdout:1769 read 0xa394200 thru 0xa3a0520 (0xc321 bytes) 2024-09-19T21:35:07.827 INFO:teuthology.orchestra.run.smithi122.stdout:1772 write 0x477c000 thru 0x478b249 (0xf24a bytes) 2024-09-19T21:35:07.828 INFO:teuthology.orchestra.run.smithi122.stdout:1773 write 0xadf2e00 thru 0xadfecf7 (0xbef8 bytes) 2024-09-19T21:35:07.828 INFO:teuthology.orchestra.run.smithi122.stdout:1774 read 0x9c59c00 thru 0x9c5a1b3 (0x5b4 bytes) 2024-09-19T21:35:07.829 INFO:teuthology.orchestra.run.smithi122.stdout:1776 write 0x6153e00 thru 0x615b036 (0x7237 bytes) 2024-09-19T21:35:07.830 INFO:teuthology.orchestra.run.smithi122.stdout:1777 write 0x9257800 thru 0x925b05e (0x385f bytes) 2024-09-19T21:35:07.830 INFO:teuthology.orchestra.run.smithi122.stdout:1778 read 0x9a38e00 thru 0x9a3987e (0xa7f bytes) 2024-09-19T21:35:07.832 INFO:teuthology.orchestra.run.smithi122.stdout:1779 read 0x55cae00 thru 0x55d3f26 (0x9127 bytes) 2024-09-19T21:35:07.833 INFO:teuthology.orchestra.run.smithi122.stdout:1781 read 0x4b32200 thru 0x4b3e990 (0xc791 bytes) 2024-09-19T21:35:07.834 INFO:teuthology.orchestra.run.smithi122.stdout:1782 read 0x8d67800 thru 0x8d7380b (0xc00c bytes) 2024-09-19T21:35:07.836 INFO:teuthology.orchestra.run.smithi122.stdout:1783 read 0xb245e00 thru 0xb247e27 (0x2028 bytes) 2024-09-19T21:35:07.836 INFO:teuthology.orchestra.run.smithi122.stdout:1784 write 0x9a68400 thru 0x9a737d1 (0xb3d2 bytes) 2024-09-19T21:35:07.837 INFO:teuthology.orchestra.run.smithi122.stdout:1786 write 0x9edfc00 thru 0x9ee2f03 (0x3304 bytes) 2024-09-19T21:35:07.838 INFO:teuthology.orchestra.run.smithi122.stdout:1788 punch from 0x61000 to 0x70c00, (0xfc00 bytes) 2024-09-19T21:35:07.843 INFO:teuthology.orchestra.run.smithi122.stdout:1789 read 0x8aa7a00 thru 0x8ab1a76 (0xa077 bytes) 2024-09-19T21:35:07.845 INFO:teuthology.orchestra.run.smithi122.stdout:1791 write 0xd02b200 thru 0xd036b04 (0xb905 bytes) 2024-09-19T21:35:07.845 INFO:teuthology.orchestra.run.smithi122.stdout:1792 read 0xcacae00 thru 0xcad12b1 (0x64b2 bytes) 2024-09-19T21:35:07.852 INFO:teuthology.orchestra.run.smithi122.stdout:1795 trunc from 0xdbd0200 to 0xb379a00 2024-09-19T21:35:07.864 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:07.861+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:07.889 INFO:teuthology.orchestra.run.smithi122.stdout:1796 punch from 0x2c30800 to 0x2c34400, (0x3c00 bytes) 2024-09-19T21:35:07.892 INFO:teuthology.orchestra.run.smithi122.stdout:1797 punch from 0x980ba00 to 0x9814800, (0x8e00 bytes) 2024-09-19T21:35:07.892 INFO:teuthology.orchestra.run.smithi122.stdout:1800 write 0x2759800 thru 0x275a056 (0x857 bytes) 2024-09-19T21:35:07.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:07.904 INFO:teuthology.orchestra.run.smithi122.stdout:1803 trunc from 0xb379a00 to 0xd148600 2024-09-19T21:35:07.937 INFO:teuthology.orchestra.run.smithi122.stdout:1804 write 0x7adec00 thru 0x7ae0bbb (0x1fbc bytes) 2024-09-19T21:35:07.939 INFO:teuthology.orchestra.run.smithi122.stdout:1806 write 0x1afac00 thru 0x1afc474 (0x1875 bytes) 2024-09-19T21:35:07.964 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:07.962+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:07.971 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:07.971 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:07.974 INFO:teuthology.orchestra.run.smithi122.stdout:1807 write 0xd61a200 thru 0xd61f7cb (0x55cc bytes) 2024-09-19T21:35:07.976 INFO:teuthology.orchestra.run.smithi122.stdout:1809 read 0x7489a00 thru 0x749712d (0xd72e bytes) 2024-09-19T21:35:07.977 INFO:teuthology.orchestra.run.smithi122.stdout:1810 read 0xae66a00 thru 0xae6ddb2 (0x73b3 bytes) 2024-09-19T21:35:07.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:07.989 INFO:teuthology.orchestra.run.smithi122.stdout:1812 trunc from 0xd61f800 to 0xe6f8a00 2024-09-19T21:35:08.046 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:08.064 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:08.061+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:08.068 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:08.073 INFO:teuthology.orchestra.run.smithi122.stdout:1815 trunc from 0xe6f8a00 to 0x12c6a00 2024-09-19T21:35:08.090 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:08.090 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:08.096 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:08.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:08.122 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:08.147 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:08.158 INFO:teuthology.orchestra.run.smithi122.stdout:1817 trunc from 0x12c6a00 to 0xad30a00 2024-09-19T21:35:08.166 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:08.161+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:08.170 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:08.173 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:08.176 INFO:teuthology.orchestra.run.smithi122.stdout:1819 write 0x126000 thru 0x12b218 (0x5219 bytes) 2024-09-19T21:35:08.181 INFO:teuthology.orchestra.run.smithi122.stdout:1821 trunc from 0xad30a00 to 0xa71d000 2024-09-19T21:35:08.199 INFO:teuthology.orchestra.run.smithi122.stdout:1824 read 0x95d5a00 thru 0x95d665c (0xc5d bytes) 2024-09-19T21:35:08.201 INFO:teuthology.orchestra.run.smithi122.stdout:1825 write 0x2546000 thru 0x25506ae (0xa6af bytes) 2024-09-19T21:35:08.201 INFO:teuthology.orchestra.run.smithi122.stdout:1827 read 0x6f31600 thru 0x6f3ec2b (0xd62c bytes) 2024-09-19T21:35:08.201 INFO:teuthology.orchestra.run.smithi122.stdout:1828 write 0x1214a00 thru 0x1220235 (0xb836 bytes) 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1075 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 7168~512] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.497958520999999, 2024-09-19T21:35:08.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030194739000000002, 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1075 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:08.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712126+0000", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5739999999999998e-06 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712134+0000", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6550000000000004e-06 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712139+0000", 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2270000000000001e-06 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742210+0000", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030070403999999998 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742253+0000", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3010000000000003e-05 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742319+0000", 2024-09-19T21:35:08.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6241000000000005e-05 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1077 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.497937834, 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030330979000000001, 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1077 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712144+0000", 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712148+0000", 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3330000000000001e-06 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712156+0000", 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3280000000000006e-06 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712161+0000", 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2299999999999999e-06 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742353+0000", 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030191913000000001 2024-09-19T21:35:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742398+0000", 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4669000000000002e-05 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742476+0000", 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8181000000000002e-05 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1480 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.477882629, 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030120475000000001, 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1480 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732201+0000", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4160000000000001e-06 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732205+0000", 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2169999999999996e-06 2024-09-19T21:35:08.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732208+0000", 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6259999999999999e-06 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762084+0000", 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029876117000000001 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762193+0000", 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108906 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762321+0000", 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012741400000000001 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1485 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.477872317999999, 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030359516999999999, 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1485 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732212+0000", 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5269999999999999e-06 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732215+0000", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.692e-06 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732218+0000", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6010000000000002e-06 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762393+0000", 2024-09-19T21:35:08.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030175415000000001 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762470+0000", 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6774999999999996e-05 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762570+0000", 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9883000000000006e-05 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1540 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473820977999999, 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.03012836, 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1540 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736261+0000", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736264+0000", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3120000000000001e-06 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736269+0000", 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:08.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736274+0000", 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.69e-06 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766159+0000", 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029884984 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766269+0000", 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110098 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766390+0000", 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121489 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1543 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.47380398, 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030340459, 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1543 2024-09-19T21:35:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736278+0000", 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736281+0000", 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4250000000000001e-06 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736286+0000", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.665e-06 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736293+0000", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5039999999999999e-06 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766437+0000", 2024-09-19T21:35:08.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030144388000000001 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766523+0000", 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5575000000000007e-05 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766619+0000", 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6299999999999996e-05 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1546 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473783730999999, 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030580726999999999, 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1546 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736300+0000", 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8369999999999999e-06 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736304+0000", 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9890000000000003e-06 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736314+0000", 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2550000000000003e-06 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766650+0000", 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030336609 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766753+0000", 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102413 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766880+0000", 2024-09-19T21:35:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012705999999999999 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1551 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473766178, 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030830699999999999, 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1551 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736316+0000", 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736322+0000", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8279999999999999e-06 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736325+0000", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.348e-06 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736328+0000", 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7020000000000001e-06 2024-09-19T21:35:08.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766911+0000", 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030582668 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767012+0000", 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101053 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767147+0000", 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135597 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1554 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473752589, 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031084074, 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1554 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736331+0000", 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5719999999999999e-06 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736334+0000", 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9050000000000001e-06 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736337+0000", 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6429999999999999e-06 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767184+0000", 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030846625999999999 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767286+0000", 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010242399999999999 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767414+0000", 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012813500000000001 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1559 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473722321, 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031279296999999998, 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1559 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:08.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736362+0000", 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.362e-06 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736364+0000", 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.695e-06 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736367+0000", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767459+0000", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031091534000000001 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767540+0000", 2024-09-19T21:35:08.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1353000000000005e-05 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767640+0000", 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9806e-05 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1562 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473712383000001, 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031533528999999998, 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1562 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:08.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736371+0000", 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.373e-06 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736374+0000", 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8480000000000001e-06 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736378+0000", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8850000000000001e-06 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767680+0000", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031301571 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767772+0000", 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2255000000000004e-05 2024-09-19T21:35:08.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767904+0000", 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181299999999999 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1569 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473671982000001, 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031758058999999998, 2024-09-19T21:35:08.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1569 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736410+0000", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736413+0000", 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6960000000000001e-06 2024-09-19T21:35:08.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736421+0000", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6399999999999997e-06 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736425+0000", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5279999999999997e-06 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767935+0000", 2024-09-19T21:35:08.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031509713000000002 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768041+0000", 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106397 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768169+0000", 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000127479 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1571 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473626219, 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031967883000000002, 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1571 2024-09-19T21:35:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736459+0000", 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736466+0000", 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5250000000000002e-06 2024-09-19T21:35:08.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736470+0000", 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768196+0000", 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031726035 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768298+0000", 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010195400000000001 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768424+0000", 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012614899999999999 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1572 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473607855000001, 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032212024999999998, 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1572 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.278 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736474+0000", 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736477+0000", 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4940000000000002e-06 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736482+0000", 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5010000000000004e-06 2024-09-19T21:35:08.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736488+0000", 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1079999999999998e-06 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768452+0000", 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031963671999999999 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768557+0000", 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104411 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768687+0000", 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000130307 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1574 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473590112, 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032396917999999997, 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1574 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736492+0000", 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736495+0000", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3640000000000002e-06 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736500+0000", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7069999999999997e-06 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736505+0000", 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5959999999999997e-06 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768731+0000", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032225666 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768798+0000", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7187000000000006e-05 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768890+0000", 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1730000000000004e-05 2024-09-19T21:35:08.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1575 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473572766, 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032552508000000001, 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1575 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736512+0000", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736518+0000", 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:08.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736523+0000", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5670000000000002e-06 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768914+0000", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032390713000000002 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768976+0000", 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2390000000000004e-05 2024-09-19T21:35:08.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769062+0000", 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6248000000000005e-05 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1578 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473521066, 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032774279000000003, 2024-09-19T21:35:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1578 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736561+0000", 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736565+0000", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.557e-06 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736572+0000", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9419999999999998e-06 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736576+0000", 2024-09-19T21:35:08.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4730000000000002e-06 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769102+0000", 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032525674999999997 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769205+0000", 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010330700000000001 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769336+0000", 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013078699999999999 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1579 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473501881000001, 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033004987999999999, 2024-09-19T21:35:08.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1579 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736580+0000", 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736583+0000", 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8090000000000001e-06 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736589+0000", 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736593+0000", 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5210000000000004e-06 2024-09-19T21:35:08.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769367+0000", 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032773434999999997 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769463+0000", 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6069000000000005e-05 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769586+0000", 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012306200000000001 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1582 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.473420825, 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033168969999999999, 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:08.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1582 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:08.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736661+0000", 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736663+0000", 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.872e-06 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736668+0000", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9280000000000001e-06 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736677+0000", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2709999999999992e-06 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769651+0000", 2024-09-19T21:35:08.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032974366999999997 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769726+0000", 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5244e-05 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769831+0000", 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104737 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.461460893, 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:08.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:08.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:08.311 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.312 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:08.312 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:08.312 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:08.314 INFO:teuthology.orchestra.run.smithi122.stdout:1830 write 0xacfba00 thru 0xacfeb04 (0x3105 bytes) 2024-09-19T21:35:08.314 INFO:teuthology.orchestra.run.smithi122.stdout:1834 trunc from 0xacfec00 to 0x62cac00 2024-09-19T21:35:08.314 INFO:teuthology.orchestra.run.smithi122.stdout:1836 trunc from 0x62cac00 to 0xe162000 2024-09-19T21:35:08.314 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:08.309+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:08.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:08.337 INFO:teuthology.orchestra.run.smithi122.stdout:1838 trunc from 0xe162000 to 0xbcb4000 2024-09-19T21:35:08.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:35:08.356 INFO:teuthology.orchestra.run.smithi122.stdout:1839 trunc from 0xbcb4000 to 0xbf11a00 2024-09-19T21:35:08.379 INFO:teuthology.orchestra.run.smithi122.stdout:1840 read 0x8ccec00 thru 0x8cd27cc (0x3bcd bytes) 2024-09-19T21:35:08.381 INFO:teuthology.orchestra.run.smithi122.stdout:1841 write 0x6a28400 thru 0x6a33fd7 (0xbbd8 bytes) 2024-09-19T21:35:08.381 INFO:teuthology.orchestra.run.smithi122.stdout:1842 read 0x7c6ac00 thru 0x7c6da97 (0x2e98 bytes) 2024-09-19T21:35:08.381 INFO:teuthology.orchestra.run.smithi122.stdout:1845 punch from 0xa3d6000 to 0xa3e2600, (0xc600 bytes) 2024-09-19T21:35:08.382 INFO:teuthology.orchestra.run.smithi122.stdout:1846 punch from 0x5df7000 to 0x5dfac00, (0x3c00 bytes) 2024-09-19T21:35:08.382 INFO:teuthology.orchestra.run.smithi122.stdout:1847 punch from 0x9e4f400 to 0x9e51e00, (0x2a00 bytes) 2024-09-19T21:35:08.382 INFO:teuthology.orchestra.run.smithi122.stdout:1848 punch from 0x1ff2200 to 0x1ff3c00, (0x1a00 bytes) 2024-09-19T21:35:08.383 INFO:teuthology.orchestra.run.smithi122.stdout:1850 read 0x9cabe00 thru 0x9cbbaac (0xfcad bytes) 2024-09-19T21:35:08.411 INFO:teuthology.orchestra.run.smithi122.stdout:1852 write 0xc3f7800 thru 0xc403e68 (0xc669 bytes) 2024-09-19T21:35:08.412 INFO:teuthology.orchestra.run.smithi122.stdout:1854 read 0x3436a00 thru 0x3438198 (0x1799 bytes) 2024-09-19T21:35:08.412 INFO:teuthology.orchestra.run.smithi122.stdout:1855 punch from 0x88a5200 to 0x88aea00, (0x9800 bytes) 2024-09-19T21:35:08.413 INFO:teuthology.orchestra.run.smithi122.stdout:1856 read 0x2444200 thru 0x2444773 (0x574 bytes) 2024-09-19T21:35:08.414 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:08.409+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:08.414 INFO:teuthology.orchestra.run.smithi122.stdout:1857 write 0x1a88800 thru 0x1a91e92 (0x9693 bytes) 2024-09-19T21:35:08.418 INFO:teuthology.orchestra.run.smithi122.stdout:1862 punch from 0x6cf4600 to 0x6d02a00, (0xe400 bytes) 2024-09-19T21:35:08.440 INFO:teuthology.orchestra.run.smithi122.stdout:1863 trunc from 0xc404000 to 0x26f3e00 2024-09-19T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:08.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:08.498 INFO:teuthology.orchestra.run.smithi122.stdout:1864 write 0x7dd1a00 thru 0x7de0342 (0xe943 bytes) 2024-09-19T21:35:08.514 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:08.509+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:08.533 INFO:teuthology.orchestra.run.smithi122.stdout:1866 write 0xa72ce00 thru 0xa736eac (0xa0ad bytes) 2024-09-19T21:35:08.534 INFO:teuthology.orchestra.run.smithi122.stdout:1867 punch from 0x7533a00 to 0x753bc00, (0x8200 bytes) 2024-09-19T21:35:08.535 INFO:teuthology.orchestra.run.smithi122.stdout:1868 read 0x1450400 thru 0x1453639 (0x323a bytes) 2024-09-19T21:35:08.536 INFO:teuthology.orchestra.run.smithi122.stdout:1869 write 0x383d400 thru 0x3840f63 (0x3b64 bytes) 2024-09-19T21:35:08.563 INFO:teuthology.orchestra.run.smithi122.stdout:1870 write 0xb758a00 thru 0xb75a0ca (0x16cb bytes) 2024-09-19T21:35:08.582 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:08.583 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:08.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:08.596 INFO:teuthology.orchestra.run.smithi122.stdout:1873 write 0xe7cec00 thru 0xe7d0c4c (0x204d bytes) 2024-09-19T21:35:08.600 INFO:teuthology.orchestra.run.smithi122.stdout:1875 write 0x2afd200 thru 0x2aff969 (0x276a bytes) 2024-09-19T21:35:08.605 INFO:teuthology.orchestra.run.smithi122.stdout:1878 trunc from 0xe7d0e00 to 0xcd14600 2024-09-19T21:35:08.615 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:08.609+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:08.640 INFO:teuthology.orchestra.run.smithi122.stdout:1879 punch from 0x3d01c00 to 0x3d11a00, (0xfe00 bytes) 2024-09-19T21:35:08.642 INFO:teuthology.orchestra.run.smithi122.stdout:1883 write 0xc75d600 thru 0xc76b318 (0xdd19 bytes) 2024-09-19T21:35:08.642 INFO:teuthology.orchestra.run.smithi122.stdout:1885 write 0x4927600 thru 0x492ddd6 (0x67d7 bytes) 2024-09-19T21:35:08.643 INFO:teuthology.orchestra.run.smithi122.stdout:1886 read 0x82cbe00 thru 0x82db4a3 (0xf6a4 bytes) 2024-09-19T21:35:08.680 INFO:teuthology.orchestra.run.smithi122.stdout:1888 write 0xea4ce00 thru 0xea4df50 (0x1151 bytes) 2024-09-19T21:35:08.683 INFO:teuthology.orchestra.run.smithi122.stdout:1889 write 0x22c6200 thru 0x22d0a7e (0xa87f bytes) 2024-09-19T21:35:08.684 INFO:teuthology.orchestra.run.smithi122.stdout:1891 write 0xc3b800 thru 0xc461d1 (0xa9d2 bytes) 2024-09-19T21:35:08.705 INFO:teuthology.orchestra.run.smithi122.stdout:1892 trunc from 0xea4e000 to 0x4fcee00 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.445670693, 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.444344224000002, 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.718 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.444312750000002, 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:08.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.444273796000001, 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.444232753000001, 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:35:08.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434980369000002, 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:35:08.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434970140000001, 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:35:08.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.742 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434960698000001, 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:35:08.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:35:08.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:08.747 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434942258, 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434931649999999, 2024-09-19T21:35:08.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:35:08.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:35:08.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434922911000001, 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434910662, 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434711847999999, 2024-09-19T21:35:08.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434693984999999, 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434677439000001, 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434661093999999, 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434644216999999, 2024-09-19T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.43462603, 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.434607500999999, 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4809.0:1592 3.1d e42/23 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head v 42'106, mlcod=42'104)", 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 12.951454075999999, 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033307211000000003, 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751782+0000", 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758281+0000", 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0064992369999999997 2024-09-19T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758315+0000", 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4561999999999999e-05 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758330+0000", 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4822e-05 2024-09-19T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758424+0000", 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3377999999999999e-05 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758462+0000", 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8732000000000001e-05 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785018+0000", 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.026556059999999999 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785090+0000", 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1971999999999995e-05 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:08.808 INFO:teuthology.orchestra.run.smithi122.stdout:1894 trunc from 0x4fcee00 to 0x2ba1200 2024-09-19T21:35:08.808 INFO:teuthology.orchestra.run.smithi122.stdout:1895 punch from 0x2ad2e00 to 0x2add000, (0xa200 bytes) 2024-09-19T21:35:08.808 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:08.809 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:08.801+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:08.823 INFO:teuthology.orchestra.run.smithi122.stdout:1896 write 0x8062400 thru 0x806a47c (0x807d bytes) 2024-09-19T21:35:08.826 INFO:teuthology.orchestra.run.smithi122.stdout:1897 write 0x7091000 thru 0x709efbc (0xdfbd bytes) 2024-09-19T21:35:08.826 INFO:teuthology.orchestra.run.smithi122.stdout:1899 read 0x4804a00 thru 0x4809348 (0x4949 bytes) 2024-09-19T21:35:08.855 INFO:teuthology.orchestra.run.smithi122.stdout:1900 write 0x8fca600 thru 0x8fcd19c (0x2b9d bytes) 2024-09-19T21:35:08.895 INFO:teuthology.orchestra.run.smithi122.stdout:1901 write 0xe7e3000 thru 0xe7f28b4 (0xf8b5 bytes) 2024-09-19T21:35:08.961 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:08.961 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:08.961 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:08.961 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:08.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:08.962 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:08.958+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1902 trunc from 0xe7f2a00 to 0x907ae00 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1904 punch from 0x8278200 to 0x8283600, (0xb400 bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1905 read 0x6c80e00 thru 0x6c85763 (0x4964 bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1906 punch from 0x6b06e00 to 0x6b0f000, (0x8200 bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1907 read 0xe59000 thru 0xe60e92 (0x7e93 bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1908 read 0x2eb0200 thru 0x2eb148b (0x128c bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1910 read 0x2a41600 thru 0x2a48cc4 (0x76c5 bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1913 read 0x4ac7800 thru 0x4ac93fa (0x1bfb bytes) 2024-09-19T21:35:08.963 INFO:teuthology.orchestra.run.smithi122.stdout:1914 read 0x8441000 thru 0x8445c3c (0x4c3d bytes) 2024-09-19T21:35:08.987 INFO:teuthology.orchestra.run.smithi122.stdout:1916 write 0xdb4b200 thru 0xdb5434a (0x914b bytes) 2024-09-19T21:35:08.990 INFO:teuthology.orchestra.run.smithi122.stdout:1919 write 0x6d5e800 thru 0x6d66f40 (0x8741 bytes) 2024-09-19T21:35:09.019 INFO:teuthology.orchestra.run.smithi122.stdout:1921 trunc from 0xdb54400 to 0x1503200 2024-09-19T21:35:09.061 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:09.058+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:09.083 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:09.087 INFO:teuthology.orchestra.run.smithi122.stdout:1922 write 0x3983800 thru 0x3984d14 (0x1515 bytes) 2024-09-19T21:35:09.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:09.108 INFO:teuthology.orchestra.run.smithi122.stdout:1925 trunc from 0x3984e00 to 0xde17000 2024-09-19T21:35:09.133 INFO:teuthology.orchestra.run.smithi122.stdout:1927 read 0x34da000 thru 0x34e44a6 (0xa4a7 bytes) 2024-09-19T21:35:09.154 INFO:teuthology.orchestra.run.smithi122.stdout:1928 trunc from 0xde17000 to 0x3c0cc00 2024-09-19T21:35:09.161 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:09.157+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:09.179 INFO:teuthology.orchestra.run.smithi122.stdout:1929 read 0x25de000 thru 0x25ed757 (0xf758 bytes) 2024-09-19T21:35:09.210 INFO:teuthology.orchestra.run.smithi122.stdout:1932 write 0x7c7b200 thru 0x7c85dce (0xabcf bytes) 2024-09-19T21:35:09.214 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:09.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 53.277201503999997, 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:09.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:09.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:09.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.221 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4788 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940975774999998, 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053208699999999998, 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4788 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269025+0000", 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269024+0000", 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269027+0000", 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0889999999999998e-06 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269032+0000", 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.31e-06 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269050+0000", 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8138000000000002e-05 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322058+0000", 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053008161999999998 2024-09-19T21:35:09.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322126+0000", 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8269999999999995e-05 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322233+0000", 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107027 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4789 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 423936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940945202999998, 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053201143999999999, 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4789 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.227 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269055+0000", 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269066+0000", 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1392e-05 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269073+0000", 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0500000000000003e-06 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269079+0000", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322103+0000", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053024486000000003 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322168+0000", 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949000000000004e-05 2024-09-19T21:35:09.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322256+0000", 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8187000000000001e-05 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4790 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940917375000001, 2024-09-19T21:35:09.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053307251999999999, 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4790 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269083+0000", 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269097+0000", 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4481000000000001e-05 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0243e-05 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0139999999999998e-06 2024-09-19T21:35:09.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322255+0000", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053142366000000003 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322323+0000", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7725999999999995e-05 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322390+0000", 2024-09-19T21:35:09.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7800999999999997e-05 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4791 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 450560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940883491000001, 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053280393000000002, 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4791 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:09.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269117+0000", 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269119+0000", 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269125+0000", 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5180000000000004e-06 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269129+0000", 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6990000000000002e-06 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322279+0000", 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053149890999999998 2024-09-19T21:35:09.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322342+0000", 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2392999999999997e-05 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322397+0000", 2024-09-19T21:35:09.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5798999999999999e-05 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4792 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 460800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940867528999998, 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053414182999999997, 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4792 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:09.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:09.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269141+0000", 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6810000000000003e-06 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3549999999999998e-06 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322416+0000", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053270709999999999 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322474+0000", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8161000000000001e-05 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322547+0000", 2024-09-19T21:35:09.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3357999999999998e-05 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4796 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940851881, 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053558802000000003, 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4796 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269149+0000", 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269148+0000", 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0439999999999998e-06 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269155+0000", 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.967e-06 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269167+0000", 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1935e-05 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322574+0000", 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053407102999999997 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322642+0000", 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7683999999999998e-05 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322707+0000", 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5421999999999998e-05 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4799 3.14 3:2d68cd5d:::rbd_data.1106b3d15a22.0000000000000001:5 [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940828882000002, 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054302238000000003, 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4799 2024-09-19T21:35:09.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269172+0000", 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269171+0000", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0429999999999999e-06 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:35:09.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269186+0000", 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9840000000000001e-06 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323330+0000", 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054144172999999997 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323399+0000", 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9543999999999997e-05 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323474+0000", 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4564e-05 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4868 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:09.255 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940810327000001, 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053685682999999998, 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4868 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:09.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269190+0000", 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269192+0000", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2639999999999998e-06 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269198+0000", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8810000000000001e-06 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1049999999999997e-06 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322728+0000", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053525831000000003 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322792+0000", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4367000000000002e-05 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322876+0000", 2024-09-19T21:35:09.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3558000000000002e-05 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4869 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940794283999999, 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053842947000000002, 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4869 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269206+0000", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269208+0000", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3879999999999998e-06 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269214+0000", 2024-09-19T21:35:09.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7529999999999998e-06 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269221+0000", 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.657e-06 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322897+0000", 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053676662 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.322966+0000", 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9085999999999996e-05 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323049+0000", 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2742000000000002e-05 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4870 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 43008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.264 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.940590512, 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053808917999999997, 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4870 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269410+0000", 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269225+0000", 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998155 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269413+0000", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018773800000000001 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269420+0000", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7399999999999998e-06 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269424+0000", 2024-09-19T21:35:09.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9389999999999999e-06 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323071+0000", 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053647087000000003 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323137+0000", 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5810999999999997e-05 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323219+0000", 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2236000000000003e-05 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4871 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 69632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939949536, 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053346843999999997, 2024-09-19T21:35:09.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4871 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270051+0000", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.269436+0000", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9993844 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270057+0000", 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00062110799999999999 2024-09-19T21:35:09.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270066+0000", 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2709999999999998e-06 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270073+0000", 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9770000000000003e-06 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323240+0000", 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053167430000000002 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323312+0000", 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1581000000000005e-05 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323398+0000", 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5939999999999994e-05 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4872 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939920994000001, 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053465749, 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4872 2024-09-19T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270080+0000", 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270079+0000", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270082+0000", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9730000000000002e-06 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270088+0000", 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1840000000000004e-06 2024-09-19T21:35:09.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270093+0000", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323425+0000", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053332349000000001 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323493+0000", 2024-09-19T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7828000000000002e-05 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323545+0000", 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2166000000000002e-05 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4873 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 106496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939903180999998, 2024-09-19T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053689477999999999, 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4873 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270097+0000", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270099+0000", 2024-09-19T21:35:09.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270106+0000", 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270111+0000", 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7520000000000004e-06 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323583+0000", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053471779999999997 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323664+0000", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1402000000000004e-05 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323787+0000", 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122404 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.282 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4874 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 116736~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939876106, 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053900483999999999, 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4874 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270124+0000", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270131+0000", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9079999999999998e-06 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270138+0000", 2024-09-19T21:35:09.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5699999999999998e-06 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270143+0000", 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2159999999999997e-06 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323826+0000", 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053683467999999998 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.323902+0000", 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5350000000000002e-05 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324025+0000", 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012334600000000001 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4875 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 143360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.287 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939840343, 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054010483999999997, 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4875 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270160+0000", 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270177+0000", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6985000000000001e-05 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270188+0000", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0993e-05 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270193+0000", 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7679999999999997e-06 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324068+0000", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053874248 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324119+0000", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1615e-05 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324171+0000", 2024-09-19T21:35:09.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440999999999998e-05 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4876 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 153600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939555954999999, 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053834246000000002, 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4876 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270445+0000", 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270198+0000", 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997535 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270450+0000", 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025216000000000003 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270460+0000", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8409999999999995e-06 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270467+0000", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6680000000000004e-06 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324197+0000", 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053730214999999998 2024-09-19T21:35:09.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324239+0000", 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1755999999999998e-05 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324279+0000", 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0224e-05 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4877 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939526677, 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054028259000000002, 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4877 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270474+0000", 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270473+0000", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270476+0000", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8590000000000001e-06 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270490+0000", 2024-09-19T21:35:09.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4071e-05 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270496+0000", 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7599999999999999e-06 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324308+0000", 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053811953000000003 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324349+0000", 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0747999999999997e-05 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324502+0000", 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015331599999999999 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.301 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4878 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 190464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.939479329000001, 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054078566000000002, 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4878 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270521+0000", 2024-09-19T21:35:09.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.270524+0000", 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271662+0000", 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001138114 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.271683+0000", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0638000000000001e-05 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324520+0000", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052837566000000002 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324559+0000", 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8782999999999998e-05 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.324600+0000", 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.066e-05 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4881 3.1c 3:3f002905:::rbd_data.1106b3d15a22.0000000000000008:5 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.937952047, 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.053152144999999998, 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.306 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4881 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272048+0000", 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272051+0000", 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8890000000000002e-06 2024-09-19T21:35:09.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272057+0000", 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6030000000000004e-06 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.272066+0000", 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5540000000000003e-06 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325024+0000", 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052957551999999998 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325114+0000", 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0708e-05 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.325201+0000", 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6236999999999994e-05 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.310 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.311 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.311 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:09.312 INFO:teuthology.orchestra.run.smithi122.stdout:1934 write 0xaf90000 thru 0xaf926bd (0x26be bytes) 2024-09-19T21:35:09.312 INFO:teuthology.orchestra.run.smithi122.stdout:1935 clone 14 order 25 su 1048576 sc 10 2024-09-19T21:35:09.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:09.313 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:09.310+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:09.350 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:35:09.413 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:09.410+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:6160 3.5 3:a461e80c:::image_client.0-clone14.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:09.433846+0000", 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.000274097, 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031663200000000001, 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 6160 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433846+0000", 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433846+0000", 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433844+0000", 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433855+0000", 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1263999999999999e-05 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433887+0000", 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2126999999999999e-05 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.433906+0000", 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8541000000000001e-05 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:09.434023+0000", 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011688000000000001 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.440 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:09.441 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:35:09.441 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:09.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:09.499 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone13 2024-09-19T21:35:09.514 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:09.509+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:09.555 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:09.555 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:09.556 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:09.556 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:09.565 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:09.615 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:09.610+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 28.445215544, 2024-09-19T21:35:09.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:09.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1446 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971445709999999, 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028575401, 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1446 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701211+0000", 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701217+0000", 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4450000000000001e-06 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701232+0000", 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5211999999999999e-05 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701236+0000", 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8689999999999997e-06 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729646+0000", 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028410086000000001 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729707+0000", 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0690999999999999e-05 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729786+0000", 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9327999999999997e-05 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.685 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1456 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971314849000001, 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028614296000000001, 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1456 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701342+0000", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701341+0000", 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:09.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701345+0000", 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6969999999999999e-06 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701351+0000", 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.122e-06 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701368+0000", 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7487999999999999e-05 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729825+0000", 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028456535000000002 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729885+0000", 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9738999999999998e-05 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729956+0000", 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1496999999999997e-05 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1457 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971284525, 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028756075999999998, 2024-09-19T21:35:09.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1457 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701372+0000", 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701374+0000", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9319999999999999e-06 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701377+0000", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.659e-06 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701380+0000", 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8650000000000001e-06 2024-09-19T21:35:09.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729994+0000", 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028613791 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730052+0000", 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8508999999999998e-05 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730128+0000", 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5683e-05 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1460 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971274354, 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028920154999999999, 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1460 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701382+0000", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701383+0000", 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2389999999999999e-06 2024-09-19T21:35:09.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701386+0000", 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.717e-06 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701389+0000", 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8990000000000001e-06 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730165+0000", 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02877569 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730227+0000", 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2791000000000002e-05 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730302+0000", 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5036999999999993e-05 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.699 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1461 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971262243, 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02910503, 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1461 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701394+0000", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701397+0000", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.576e-06 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701402+0000", 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4160000000000003e-06 2024-09-19T21:35:09.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701407+0000", 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730338+0000", 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028931354999999999 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730404+0000", 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5630999999999996e-05 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730499+0000", 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5397999999999997e-05 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.704 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971187735999999, 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:09.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:09.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971157909, 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:09.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971148223, 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971139897, 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.718 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:09.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:09.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971129505, 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.723 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:09.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.971119009000001, 2024-09-19T21:35:09.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.731 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1530 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968863035, 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028447986000000001, 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1530 2024-09-19T21:35:09.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703794+0000", 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703793+0000", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703807+0000", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4080000000000001e-05 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703816+0000", 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9670000000000007e-06 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703822+0000", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732054+0000", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028232126999999999 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732131+0000", 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7206000000000006e-05 2024-09-19T21:35:09.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732242+0000", 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110623 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1531 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:09.736 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968829835999999, 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028589459000000001, 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1531 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:09.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703827+0000", 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703826+0000", 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703835+0000", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9910000000000008e-06 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703844+0000", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9390000000000003e-06 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703849+0000", 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.147e-06 2024-09-19T21:35:09.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732283+0000", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028433429999999999 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732339+0000", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5804999999999998e-05 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732416+0000", 2024-09-19T21:35:09.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7625000000000003e-05 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1534 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968802229, 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028735752, 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1534 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:09.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703854+0000", 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703856+0000", 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4669999999999998e-06 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703866+0000", 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2909999999999996e-06 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703879+0000", 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3698e-05 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732452+0000", 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028572441000000001 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732514+0000", 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1785999999999994e-05 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732590+0000", 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6440000000000007e-05 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1541 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968772677, 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028868288999999998, 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1541 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703884+0000", 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703890+0000", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8199999999999999e-06 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703897+0000", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703902+0000", 2024-09-19T21:35:09.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1170000000000003e-06 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732626+0000", 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028723790999999999 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732683+0000", 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6645999999999999e-05 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732752+0000", 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9426999999999998e-05 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1544 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968725686999999, 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028982658000000001, 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1544 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703931+0000", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703930+0000", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703933+0000", 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7539999999999998e-06 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703939+0000", 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4130000000000003e-06 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703945+0000", 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0159999999999999e-06 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732784+0000", 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028839773999999999 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732841+0000", 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6759000000000003e-05 2024-09-19T21:35:09.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732914+0000", 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2564999999999994e-05 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1545 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968704229, 2024-09-19T21:35:09.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029130377999999998, 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1545 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703952+0000", 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703955+0000", 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6860000000000002e-06 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703968+0000", 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3226e-05 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703974+0000", 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9510000000000003e-06 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.732951+0000", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028976842999999999 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733005+0000", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4842000000000001e-05 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733083+0000", 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7380999999999997e-05 2024-09-19T21:35:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.968674921, 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:09.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:09.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.943457535, 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:09.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:09.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:09.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:09.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:09.772 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:09.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:09.774 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:09.770+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:09.875 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:09.869+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:09.900 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:09.901 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:09.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:09.976 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:09.974+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:10.021 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:10.022 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:10.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:10.077 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:10.073+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.144 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4296 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 272384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886299527999999, 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022203509999999999, 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4296 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253830+0000", 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253833+0000", 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.213e-06 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253840+0000", 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5930000000000002e-06 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253845+0000", 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3979999999999998e-06 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275901+0000", 2024-09-19T21:35:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022056172999999998 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.275964+0000", 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2723000000000002e-05 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276034+0000", 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9774e-05 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4297 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886279668, 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359332999999999, 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4297 2024-09-19T21:35:10.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:10.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253850+0000", 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253852+0000", 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4779999999999998e-06 2024-09-19T21:35:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253858+0000", 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7740000000000001e-06 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253860+0000", 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6570000000000001e-06 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276057+0000", 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022196344999999999 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276120+0000", 2024-09-19T21:35:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3521000000000004e-05 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276209+0000", 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9135999999999995e-05 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4298 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 309248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886265118000001, 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022554268999999998, 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4298 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253865+0000", 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253864+0000", 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253873+0000", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0289999999999994e-06 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253880+0000", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4729999999999997e-06 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253888+0000", 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:35:10.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276235+0000", 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022346462000000001 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276323+0000", 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7990000000000003e-05 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276419+0000", 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6291000000000004e-05 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4299 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 335872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886227519999998, 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022708860000000001, 2024-09-19T21:35:10.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4299 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253902+0000", 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253905+0000", 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.591e-06 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253921+0000", 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5282e-05 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253927+0000", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.003e-06 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276468+0000", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022541017 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276535+0000", 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7119000000000006e-05 2024-09-19T21:35:10.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276611+0000", 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6347000000000006e-05 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4300 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 346112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886199836999999, 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022876461000000001, 2024-09-19T21:35:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4300 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253930+0000", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253932+0000", 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.857e-06 2024-09-19T21:35:10.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253935+0000", 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253938+0000", 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.441e-06 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276636+0000", 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022698381 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276702+0000", 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5258000000000004e-05 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276806+0000", 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010488399999999999 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4301 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 372736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.886189838, 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023114603000000001, 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4301 2024-09-19T21:35:10.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253940+0000", 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253941+0000", 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.44e-06 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253944+0000", 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0960000000000001e-06 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.253947+0000", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4109999999999999e-06 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276839+0000", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022892550000000001 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.276947+0000", 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107793 2024-09-19T21:35:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277055+0000", 2024-09-19T21:35:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010752299999999999 2024-09-19T21:35:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4302 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 382976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.175 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885955876000001, 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023063824, 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4302 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254174+0000", 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254173+0000", 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254179+0000", 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0789999999999997e-06 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254190+0000", 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0532e-05 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254196+0000", 2024-09-19T21:35:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6560000000000003e-06 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277079+0000", 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022882731 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277146+0000", 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6934000000000007e-05 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277238+0000", 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1891999999999994e-05 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4303 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 409600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885927301999999, 2024-09-19T21:35:10.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023222402, 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4303 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254202+0000", 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254205+0000", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8329999999999998e-06 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254212+0000", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6549999999999998e-06 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254217+0000", 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277263+0000", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023045955 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277333+0000", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0257000000000002e-05 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277425+0000", 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1952000000000003e-05 2024-09-19T21:35:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4304 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 419840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885884217000001, 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023378263, 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4304 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254246+0000", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254245+0000", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254248+0000", 2024-09-19T21:35:10.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2940000000000001e-06 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254256+0000", 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254261+0000", 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5310000000000001e-06 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277449+0000", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023187433 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277538+0000", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9474000000000004e-05 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277624+0000", 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5560000000000001e-05 2024-09-19T21:35:10.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4305 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 446464~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885859398000001, 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033521316000000002, 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4305 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254270+0000", 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254273+0000", 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0850000000000001e-06 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254280+0000", 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0720000000000002e-06 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254285+0000", 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3839999999999996e-06 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277648+0000", 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0233627 2024-09-19T21:35:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.277727+0000", 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9128999999999994e-05 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.287792+0000", 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.010064559000000001 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4307 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885678867999999, 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027454658999999999, 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4307 2024-09-19T21:35:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254451+0000", 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254450+0000", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254455+0000", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4560000000000002e-06 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254464+0000", 2024-09-19T21:35:10.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8570000000000007e-06 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.254471+0000", 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9009999999999997e-06 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281760+0000", 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027288818999999999 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281834+0000", 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4019999999999999e-05 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281906+0000", 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1353999999999995e-05 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.202 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4309 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 483328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.885100018999999, 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027050399999999999, 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4309 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255030+0000", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255029+0000", 2024-09-19T21:35:10.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255035+0000", 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6500000000000001e-06 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255046+0000", 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1382999999999999e-05 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255061+0000", 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.554e-05 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.281929+0000", 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.02686796 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282019+0000", 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9860999999999999e-05 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282080+0000", 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0891999999999999e-05 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4311 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 493568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.884917882, 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027091388000000001, 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4311 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255212+0000", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255211+0000", 2024-09-19T21:35:10.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255216+0000", 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3319999999999999e-06 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255228+0000", 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1202999999999999e-05 2024-09-19T21:35:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255235+0000", 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0350000000000004e-06 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282101+0000", 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.026866476 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282202+0000", 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010135 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282303+0000", 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100883 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4313 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 520192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.884878907000001, 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027229189000000001, 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4313 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255251+0000", 2024-09-19T21:35:10.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255250+0000", 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255254+0000", 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0629999999999999e-06 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255262+0000", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-06 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.255269+0000", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4050000000000003e-06 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282326+0000", 2024-09-19T21:35:10.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.027057184000000001 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282403+0000", 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6720999999999999e-05 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.282480+0000", 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7001000000000004e-05 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4744 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 434176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.871040218000001, 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022372296999999999, 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4744 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269090+0000", 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269089+0000", 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269092+0000", 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269102+0000", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0077e-05 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269107+0000", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0289999999999996e-06 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291277+0000", 2024-09-19T21:35:10.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022170091999999999 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291356+0000", 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9241999999999998e-05 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291462+0000", 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105609 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4746 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 444416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.871018514999999, 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022538171999999999, 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4746 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269111+0000", 2024-09-19T21:35:10.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269113+0000", 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6070000000000002e-06 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269123+0000", 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0820000000000005e-06 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269128+0000", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0549999999999999e-06 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291488+0000", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022359917 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291558+0000", 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0339000000000005e-05 2024-09-19T21:35:10.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291649+0000", 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1558000000000007e-05 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4747 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 471040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.870996867999999, 2024-09-19T21:35:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022815518999999999, 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4747 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269133+0000", 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269135+0000", 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269145+0000", 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0185e-05 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269150+0000", 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7169999999999999e-06 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291712+0000", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022561748 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291806+0000", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3826999999999995e-05 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291948+0000", 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014276600000000001 2024-09-19T21:35:10.231 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4749 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.870975620999999, 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023058227000000001, 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4749 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269154+0000", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269156+0000", 2024-09-19T21:35:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2230000000000001e-06 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269165+0000", 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8470000000000002e-06 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269170+0000", 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8489999999999997e-06 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.291977+0000", 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.022806918999999998 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1191000000000003e-05 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292212+0000", 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016450899999999999 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4751 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.870955746, 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023246788000000001, 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4751 2024-09-19T21:35:10.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269174+0000", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269181+0000", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6450000000000005e-06 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269191+0000", 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9550000000000004e-06 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269196+0000", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8500000000000002e-06 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292239+0000", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023042450999999999 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292360+0000", 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121804 2024-09-19T21:35:10.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292421+0000", 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0467000000000002e-05 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4554.0:4753 3.18 3:1cc94784:::rbd_data.1106b3d15a22.0000000000000000:5 [read 518144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 28.870927890000001, 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023417911999999999, 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4554", 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4753 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:10.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269202+0000", 2024-09-19T21:35:10.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269204+0000", 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.481e-06 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269213+0000", 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5000000000000005e-06 2024-09-19T21:35:10.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.269219+0000", 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1919999999999996e-06 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292445+0000", 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.023226009999999998 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292517+0000", 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1898999999999997e-05 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:41.292620+0000", 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103198 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:10.246 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:10.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:10.248 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-19T21:35:10.249 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:10.246+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:10.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:10.309 INFO:teuthology.orchestra.run.smithi122.stdout:1938 read 0x8fed600 thru 0x8ff131d (0x3d1e bytes) 2024-09-19T21:35:10.310 INFO:teuthology.orchestra.run.smithi122.stdout:1939 write 0x7c8b000 thru 0x7c8b8f1 (0x8f2 bytes) 2024-09-19T21:35:10.311 INFO:teuthology.orchestra.run.smithi122.stdout:1940 punch from 0x5916200 to 0x591da00, (0x7800 bytes) 2024-09-19T21:35:10.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:10.333 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:10.333 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-19T21:35:10.333 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:35:10.333 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:35:10.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-19T21:35:10.349 INFO:teuthology.orchestra.run.smithi122.stdout:1941 read 0xa3ac600 thru 0xa3babbb (0xe5bc bytes) 2024-09-19T21:35:10.349 INFO:teuthology.orchestra.run.smithi122.stdout:1942 punch from 0x4d91600 to 0x4da0e00, (0xf800 bytes) 2024-09-19T21:35:10.349 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:10.345+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:10.358 INFO:teuthology.orchestra.run.smithi122.stdout:1943 read 0x6409800 thru 0x641360b (0x9e0c bytes) 2024-09-19T21:35:10.374 INFO:teuthology.orchestra.run.smithi122.stdout:1951 trunc from 0xaf92800 to 0xc32ac00 2024-09-19T21:35:10.395 INFO:teuthology.orchestra.run.smithi122.stdout:1952 write 0xb393000 thru 0xb397fe2 (0x4fe3 bytes) 2024-09-19T21:35:10.396 INFO:teuthology.orchestra.run.smithi122.stdout:1955 write 0x36d5200 thru 0x36e008f (0xae90 bytes) 2024-09-19T21:35:10.397 INFO:teuthology.orchestra.run.smithi122.stdout:1956 read 0xb23a400 thru 0xb24868d (0xe28e bytes) 2024-09-19T21:35:10.398 INFO:teuthology.orchestra.run.smithi122.stdout:1957 write 0x1450800 thru 0x145480f (0x4010 bytes) 2024-09-19T21:35:10.399 INFO:teuthology.orchestra.run.smithi122.stdout:1959 write 0x9693e00 thru 0x96a337a (0xf57b bytes) 2024-09-19T21:35:10.399 INFO:teuthology.orchestra.run.smithi122.stdout:1960 read 0x1521200 thru 0x1530de0 (0xfbe1 bytes) 2024-09-19T21:35:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:1961 write 0x6daf600 thru 0x6db817b (0x8b7c bytes) 2024-09-19T21:35:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:1962 write 0x5a0ae00 thru 0x5a0f189 (0x438a bytes) 2024-09-19T21:35:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:1964 punch from 0xc14c600 to 0xc158e00, (0xc800 bytes) 2024-09-19T21:35:10.402 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:10.405 INFO:teuthology.orchestra.run.smithi122.stdout:1966 write 0x697b200 thru 0x69836e9 (0x84ea bytes) 2024-09-19T21:35:10.408 INFO:teuthology.orchestra.run.smithi122.stdout:1969 read 0x7259000 thru 0x725f0bb (0x60bc bytes) 2024-09-19T21:35:10.409 INFO:teuthology.orchestra.run.smithi122.stdout:1970 read 0x18fd400 thru 0x1905ddc (0x89dd bytes) 2024-09-19T21:35:10.410 INFO:teuthology.orchestra.run.smithi122.stdout:1972 read 0x334400 thru 0x340e97 (0xca98 bytes) 2024-09-19T21:35:10.411 INFO:teuthology.orchestra.run.smithi122.stdout:1975 punch from 0xc062000 to 0xc070c00, (0xec00 bytes) 2024-09-19T21:35:10.414 INFO:teuthology.orchestra.run.smithi122.stdout:1978 write 0x1b8dc00 thru 0x1b8fa15 (0x1e16 bytes) 2024-09-19T21:35:10.416 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:10.416 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:10.416 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:10.416 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:10.426 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:10.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:10.445 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:10.447 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:10.447 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T21:35:10.447 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:10.450 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:10.446+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:10.465 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:10.468 INFO:teuthology.orchestra.run.smithi122.stdout:1980 trunc from 0xc32ac00 to 0x32bf400 2024-09-19T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:10.515 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:10.538 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:10.538 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:10.545 INFO:teuthology.orchestra.run.smithi122.stdout:1981 write 0x76a000 thru 0x779c19 (0xfc1a bytes) 2024-09-19T21:35:10.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:10.551 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:10.546+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:10.563 INFO:teuthology.orchestra.run.smithi122.stdout:1982 trunc from 0x32bf400 to 0x9fc7400 2024-09-19T21:35:10.610 INFO:teuthology.orchestra.run.smithi122.stdout:1984 trunc from 0x9fc7400 to 0xe4c8400 2024-09-19T21:35:10.630 INFO:teuthology.orchestra.run.smithi122.stdout:1985 punch from 0x8882800 to 0x888f000, (0xc800 bytes) 2024-09-19T21:35:10.631 INFO:teuthology.orchestra.run.smithi122.stdout:1986 write 0x412a200 thru 0x4131d0c (0x7b0d bytes) 2024-09-19T21:35:10.632 INFO:teuthology.orchestra.run.smithi122.stdout:1987 read 0xa09e600 thru 0xa0a130c (0x2d0d bytes) 2024-09-19T21:35:10.634 INFO:teuthology.orchestra.run.smithi122.stdout:1990 read 0x297d400 thru 0x2980772 (0x3373 bytes) 2024-09-19T21:35:10.636 INFO:teuthology.orchestra.run.smithi122.stdout:1991 read 0x9898e00 thru 0x989c88f (0x3a90 bytes) 2024-09-19T21:35:10.637 INFO:teuthology.orchestra.run.smithi122.stdout:1992 read 0x7226a00 thru 0x72287f9 (0x1dfa bytes) 2024-09-19T21:35:10.638 INFO:teuthology.orchestra.run.smithi122.stdout:1993 punch from 0xc9d9400 to 0xc9e4200, (0xae00 bytes) 2024-09-19T21:35:10.639 INFO:teuthology.orchestra.run.smithi122.stdout:1994 write 0xcefb800 thru 0xcefd52b (0x1d2c bytes) 2024-09-19T21:35:10.652 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:10.650+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:10.656 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:10.665 INFO:teuthology.orchestra.run.smithi122.stdout:1995 write 0xe84bc00 thru 0xe8578e5 (0xbce6 bytes) 2024-09-19T21:35:10.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:10.667 INFO:teuthology.orchestra.run.smithi122.stdout:1996 read 0xc80c00 thru 0xc828d7 (0x1cd8 bytes) 2024-09-19T21:35:10.668 INFO:teuthology.orchestra.run.smithi122.stdout:1997 write 0x5c4b000 thru 0x5c5552f (0xa530 bytes) 2024-09-19T21:35:10.669 INFO:teuthology.orchestra.run.smithi122.stdout:1998 read 0x8e73200 thru 0x8e78ac2 (0x58c3 bytes) 2024-09-19T21:35:10.670 INFO:teuthology.orchestra.run.smithi122.stdout:1999 read 0x246200 thru 0x253f56 (0xdd57 bytes) 2024-09-19T21:35:10.671 INFO:teuthology.orchestra.run.smithi122.stdout:2000 punch from 0x1aaaa00 to 0x1aab400, (0xa00 bytes) 2024-09-19T21:35:10.673 INFO:teuthology.orchestra.run.smithi122.stdout:2002 write 0x9d1cc00 thru 0x9d2ca23 (0xfe24 bytes) 2024-09-19T21:35:10.721 INFO:teuthology.orchestra.run.smithi122.stdout:2003 trunc from 0xe857a00 to 0x180000 2024-09-19T21:35:10.753 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:10.749+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:10.776 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:10.776 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:10.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:10.854 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:10.849+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:10.869 INFO:teuthology.orchestra.run.smithi122.stdout:2005 trunc from 0x180000 to 0xd94aa00 2024-09-19T21:35:10.891 INFO:teuthology.orchestra.run.smithi122.stdout:2006 punch from 0x71fb200 to 0x720b000, (0xfe00 bytes) 2024-09-19T21:35:10.893 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:10.893 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:10.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:10.905 INFO:teuthology.orchestra.run.smithi122.stdout:2009 trunc from 0xd94aa00 to 0x8572a00 2024-09-19T21:35:10.927 INFO:teuthology.orchestra.run.smithi122.stdout:2011 read 0x22f6a00 thru 0x2300f0b (0xa50c bytes) 2024-09-19T21:35:10.955 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:10.950+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:10.963 INFO:teuthology.orchestra.run.smithi122.stdout:2016 write 0xd72fc00 thru 0xd735099 (0x549a bytes) 2024-09-19T21:35:10.965 INFO:teuthology.orchestra.run.smithi122.stdout:2018 write 0xc5f2400 thru 0xc5f6587 (0x4188 bytes) 2024-09-19T21:35:10.967 INFO:teuthology.orchestra.run.smithi122.stdout:2019 trunc from 0xd735200 to 0xc964a00 2024-09-19T21:35:10.999 INFO:teuthology.orchestra.run.smithi122.stdout:2020 write 0x84faa00 thru 0x84fe688 (0x3c89 bytes) 2024-09-19T21:35:11.036 INFO:teuthology.orchestra.run.smithi122.stdout:2022 write 0xeaac200 thru 0xeab8cf4 (0xcaf5 bytes) 2024-09-19T21:35:11.038 INFO:teuthology.orchestra.run.smithi122.stdout:2027 read 0x6992400 thru 0x699dd3b (0xb93c bytes) 2024-09-19T21:35:11.039 INFO:teuthology.orchestra.run.smithi122.stdout:2030 read 0x9e5fa00 thru 0x9e67dc9 (0x83ca bytes) 2024-09-19T21:35:11.056 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:11.054+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:11.064 INFO:teuthology.orchestra.run.smithi122.stdout:2031 trunc from 0xeab8e00 to 0x1f79200 2024-09-19T21:35:11.077 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.077 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:11.153 INFO:teuthology.orchestra.run.smithi122.stdout:2032 write 0xceaa800 thru 0xceb6336 (0xbb37 bytes) 2024-09-19T21:35:11.155 INFO:teuthology.orchestra.run.smithi122.stdout:2033 punch from 0xc903800 to 0xc912800, (0xf000 bytes) 2024-09-19T21:35:11.155 INFO:teuthology.orchestra.run.smithi122.stdout:2035 read 0x3849800 thru 0x3855d31 (0xc532 bytes) 2024-09-19T21:35:11.157 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:11.153+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:11.180 INFO:teuthology.orchestra.run.smithi122.stdout:2037 write 0xe213400 thru 0xe216e8a (0x3a8b bytes) 2024-09-19T21:35:11.185 INFO:teuthology.orchestra.run.smithi122.stdout:2040 read 0x5a33200 thru 0x5a35401 (0x2202 bytes) 2024-09-19T21:35:11.195 INFO:teuthology.orchestra.run.smithi122.stdout:2041 trunc from 0xe217000 to 0x89a7a00 2024-09-19T21:35:11.216 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.216 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:11.230 INFO:teuthology.orchestra.run.smithi122.stdout:2042 read 0x1882400 thru 0x1888fd4 (0x6bd5 bytes) 2024-09-19T21:35:11.235 INFO:teuthology.orchestra.run.smithi122.stdout:2043 read 0x691ec00 thru 0x692e850 (0xfc51 bytes) 2024-09-19T21:35:11.257 INFO:teuthology.orchestra.run.smithi122.stdout:2048 write 0x9dba600 thru 0x9dc2c5b (0x865c bytes) 2024-09-19T21:35:11.258 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:11.254+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:11.259 INFO:teuthology.orchestra.run.smithi122.stdout:2050 read 0xe25000 thru 0xe2b690 (0x6691 bytes) 2024-09-19T21:35:11.260 INFO:teuthology.orchestra.run.smithi122.stdout:2051 write 0x64f8400 thru 0x6507ecc (0xfacd bytes) 2024-09-19T21:35:11.262 INFO:teuthology.orchestra.run.smithi122.stdout:2052 read 0x9945c00 thru 0x99504cf (0xa8d0 bytes) 2024-09-19T21:35:11.301 INFO:teuthology.orchestra.run.smithi122.stdout:2053 write 0xde6fa00 thru 0xde6fefc (0x4fd bytes) 2024-09-19T21:35:11.304 INFO:teuthology.orchestra.run.smithi122.stdout:2055 write 0x8073400 thru 0x807ae4d (0x7a4e bytes) 2024-09-19T21:35:11.304 INFO:teuthology.orchestra.run.smithi122.stdout:2059 read 0xa4fde00 thru 0xa502e4c (0x504d bytes) 2024-09-19T21:35:11.305 INFO:teuthology.orchestra.run.smithi122.stdout:2060 read 0xd8a2600 thru 0xd8a2d34 (0x735 bytes) 2024-09-19T21:35:11.305 INFO:teuthology.orchestra.run.smithi122.stdout:2062 punch from 0x1b87c00 to 0x1b89800, (0x1c00 bytes) 2024-09-19T21:35:11.306 INFO:teuthology.orchestra.run.smithi122.stdout:2063 write 0x173e600 thru 0x173f360 (0xd61 bytes) 2024-09-19T21:35:11.309 INFO:teuthology.orchestra.run.smithi122.stdout:2065 punch from 0x6cf0e00 to 0x6cf7a00, (0x6c00 bytes) 2024-09-19T21:35:11.310 INFO:teuthology.orchestra.run.smithi122.stdout:2066 write 0x80ab400 thru 0x80b12d1 (0x5ed2 bytes) 2024-09-19T21:35:11.310 INFO:teuthology.orchestra.run.smithi122.stdout:2067 punch from 0xc385c00 to 0xc38ee00, (0x9200 bytes) 2024-09-19T21:35:11.331 INFO:teuthology.orchestra.run.smithi122.stdout:2068 trunc from 0xde70000 to 0x4c3a000 2024-09-19T21:35:11.341 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.341 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:11.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:35:11.370 INFO:teuthology.orchestra.run.smithi122.stdout:2070 punch from 0x2720600 to 0x2722e00, (0x2800 bytes) 2024-09-19T21:35:11.396 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:11.394+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:11.396 INFO:teuthology.orchestra.run.smithi122.stdout:2071 write 0x707be00 thru 0x7086575 (0xa776 bytes) 2024-09-19T21:35:11.399 INFO:teuthology.orchestra.run.smithi122.stdout:2075 read 0x65d4000 thru 0x65e1dfb (0xddfc bytes) 2024-09-19T21:35:11.409 INFO:teuthology.orchestra.run.smithi122.stdout:2076 trunc from 0x7086600 to 0x3debc00 2024-09-19T21:35:11.453 INFO:teuthology.orchestra.run.smithi122.stdout:2082 trunc from 0x3debc00 to 0xbce6000 2024-09-19T21:35:11.459 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:11.454+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:11.471 INFO:teuthology.orchestra.run.smithi122.stdout:2083 read 0x85c7000 thru 0x85cce2e (0x5e2f bytes) 2024-09-19T21:35:11.471 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.471 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:11.494 INFO:teuthology.orchestra.run.smithi122.stdout:2084 write 0xe2d4000 thru 0xe2e02d2 (0xc2d3 bytes) 2024-09-19T21:35:11.497 INFO:teuthology.orchestra.run.smithi122.stdout:2085 punch from 0xcc19600 to 0xcc1e000, (0x4a00 bytes) 2024-09-19T21:35:11.498 INFO:teuthology.orchestra.run.smithi122.stdout:2087 write 0xd4ac00 thru 0xd50654 (0x5a55 bytes) 2024-09-19T21:35:11.499 INFO:teuthology.orchestra.run.smithi122.stdout:2090 write 0x2004a00 thru 0x2007d30 (0x3331 bytes) 2024-09-19T21:35:11.500 INFO:teuthology.orchestra.run.smithi122.stdout:2092 write 0x8345200 thru 0x8346e74 (0x1c75 bytes) 2024-09-19T21:35:11.501 INFO:teuthology.orchestra.run.smithi122.stdout:2094 read 0x3555800 thru 0x3562847 (0xd048 bytes) 2024-09-19T21:35:11.502 INFO:teuthology.orchestra.run.smithi122.stdout:2098 read 0x61fc800 thru 0x62097ee (0xcfef bytes) 2024-09-19T21:35:11.504 INFO:teuthology.orchestra.run.smithi122.stdout:2102 write 0xc999a00 thru 0xc9a25b3 (0x8bb4 bytes) 2024-09-19T21:35:11.504 INFO:teuthology.orchestra.run.smithi122.stdout:2103 punch from 0x49c1200 to 0x49caa00, (0x9800 bytes) 2024-09-19T21:35:11.506 INFO:teuthology.orchestra.run.smithi122.stdout:2105 read 0xa35fe00 thru 0xa3641bf (0x43c0 bytes) 2024-09-19T21:35:11.512 INFO:teuthology.orchestra.run.smithi122.stdout:2108 punch from 0x3a5c400 to 0x3a6a800, (0xe400 bytes) 2024-09-19T21:35:11.512 INFO:teuthology.orchestra.run.smithi122.stdout:2109 punch from 0x1962600 to 0x1969200, (0x6c00 bytes) 2024-09-19T21:35:11.534 INFO:teuthology.orchestra.run.smithi122.stdout:2112 write 0xe656c00 thru 0xe6630e2 (0xc4e3 bytes) 2024-09-19T21:35:11.536 INFO:teuthology.orchestra.run.smithi122.stdout:2113 read 0xb96000 thru 0xba3049 (0xd04a bytes) 2024-09-19T21:35:11.538 INFO:teuthology.orchestra.run.smithi122.stdout:2116 read 0xd174e00 thru 0xd17feae (0xb0af bytes) 2024-09-19T21:35:11.539 INFO:teuthology.orchestra.run.smithi122.stdout:2118 write 0x46bc200 thru 0x46c4e81 (0x8c82 bytes) 2024-09-19T21:35:11.540 INFO:teuthology.orchestra.run.smithi122.stdout:2119 read 0xde46a00 thru 0xde47f30 (0x1531 bytes) 2024-09-19T21:35:11.540 INFO:teuthology.orchestra.run.smithi122.stdout:2121 read 0x751b600 thru 0x7521ac2 (0x64c3 bytes) 2024-09-19T21:35:11.542 INFO:teuthology.orchestra.run.smithi122.stdout:2122 write 0x98a6200 thru 0x98b0fc4 (0xadc5 bytes) 2024-09-19T21:35:11.543 INFO:teuthology.orchestra.run.smithi122.stdout:2123 write 0x57a9000 thru 0x57b8b84 (0xfb85 bytes) 2024-09-19T21:35:11.544 INFO:teuthology.orchestra.run.smithi122.stdout:2124 punch from 0x9a21a00 to 0x9a29600, (0x7c00 bytes) 2024-09-19T21:35:11.546 INFO:teuthology.orchestra.run.smithi122.stdout:2126 read 0xd8d9a00 thru 0xd8dc5a3 (0x2ba4 bytes) 2024-09-19T21:35:11.547 INFO:teuthology.orchestra.run.smithi122.stdout:2127 punch from 0x2f54400 to 0x2f63e00, (0xfa00 bytes) 2024-09-19T21:35:11.547 INFO:teuthology.orchestra.run.smithi122.stdout:2129 punch from 0x99b5800 to 0x99c4e00, (0xf600 bytes) 2024-09-19T21:35:11.547 INFO:teuthology.orchestra.run.smithi122.stdout:2133 read 0x3589600 thru 0x358b996 (0x2397 bytes) 2024-09-19T21:35:11.555 INFO:teuthology.orchestra.run.smithi122.stdout:2134 read 0x21a1a00 thru 0x21aa189 (0x878a bytes) 2024-09-19T21:35:11.556 INFO:teuthology.orchestra.run.smithi122.stdout:2136 read 0x3f22e00 thru 0x3f24bb0 (0x1db1 bytes) 2024-09-19T21:35:11.560 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:11.558+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:11.568 INFO:teuthology.orchestra.run.smithi122.stdout:2141 trunc from 0xe663200 to 0x9e6f000 2024-09-19T21:35:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.598 INFO:teuthology.orchestra.run.smithi122.stdout:2143 write 0x810ac00 thru 0x8112407 (0x7808 bytes) 2024-09-19T21:35:11.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:11.607 INFO:teuthology.orchestra.run.smithi122.stdout:2144 trunc from 0x9e6f000 to 0xc069800 2024-09-19T21:35:11.627 INFO:teuthology.orchestra.run.smithi122.stdout:2146 write 0xde6e00 thru 0xdf4b3f (0xdd40 bytes) 2024-09-19T21:35:11.644 INFO:teuthology.orchestra.run.smithi122.stdout:2148 trunc from 0xc069800 to 0x46da600 2024-09-19T21:35:11.661 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:11.657+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:11.685 INFO:teuthology.orchestra.run.smithi122.stdout:2149 punch from 0x1908400 to 0x190d600, (0x5200 bytes) 2024-09-19T21:35:11.698 INFO:teuthology.orchestra.run.smithi122.stdout:2151 trunc from 0x46da600 to 0x9cb1400 2024-09-19T21:35:11.715 INFO:teuthology.orchestra.run.smithi122.stdout:2152 read 0x5382a00 thru 0x5384493 (0x1a94 bytes) 2024-09-19T21:35:11.718 INFO:teuthology.orchestra.run.smithi122.stdout:2153 punch from 0x8985000 to 0x8994a00, (0xfa00 bytes) 2024-09-19T21:35:11.718 INFO:teuthology.orchestra.run.smithi122.stdout:2155 read 0x9091800 thru 0x909a2a4 (0x8aa5 bytes) 2024-09-19T21:35:11.719 INFO:teuthology.orchestra.run.smithi122.stdout:2157 write 0x93cf600 thru 0x93d01b0 (0xbb1 bytes) 2024-09-19T21:35:11.721 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.721 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.732 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:11.751 INFO:teuthology.orchestra.run.smithi122.stdout:2158 write 0xbef3200 thru 0xbefe50d (0xb30e bytes) 2024-09-19T21:35:11.754 INFO:teuthology.orchestra.run.smithi122.stdout:2162 punch from 0x7c9de00 to 0x7cad800, (0xfa00 bytes) 2024-09-19T21:35:11.755 INFO:teuthology.orchestra.run.smithi122.stdout:2164 punch from 0x5b36400 to 0x5b36600, (0x200 bytes) 2024-09-19T21:35:11.775 INFO:teuthology.orchestra.run.smithi122.stdout:2165 write 0xccb1e00 thru 0xccb4b20 (0x2d21 bytes) 2024-09-19T21:35:11.776 INFO:teuthology.orchestra.run.smithi122.stdout:2169 read 0xa14cc00 thru 0xa158faa (0xc3ab bytes) 2024-09-19T21:35:11.779 INFO:teuthology.orchestra.run.smithi122.stdout:2174 read 0x569f400 thru 0x56a71a7 (0x7da8 bytes) 2024-09-19T21:35:11.780 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:11.778+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:11.780 INFO:teuthology.orchestra.run.smithi122.stdout:2175 punch from 0x951c600 to 0x9524400, (0x7e00 bytes) 2024-09-19T21:35:11.781 INFO:teuthology.orchestra.run.smithi122.stdout:2176 read 0x7d96600 thru 0x7d9771a (0x111b bytes) 2024-09-19T21:35:11.785 INFO:teuthology.orchestra.run.smithi122.stdout:2177 trunc from 0xccb4c00 to 0xb4cf400 2024-09-19T21:35:11.806 INFO:teuthology.orchestra.run.smithi122.stdout:2181 write 0x101c800 thru 0x1021323 (0x4b24 bytes) 2024-09-19T21:35:11.808 INFO:teuthology.orchestra.run.smithi122.stdout:2183 write 0x4ab2400 thru 0x4ac13e0 (0xefe1 bytes) 2024-09-19T21:35:11.809 INFO:teuthology.orchestra.run.smithi122.stdout:2184 read 0x87d400 thru 0x889779 (0xc37a bytes) 2024-09-19T21:35:11.810 INFO:teuthology.orchestra.run.smithi122.stdout:2185 write 0x50c8600 thru 0x50cf5b2 (0x6fb3 bytes) 2024-09-19T21:35:11.843 INFO:teuthology.orchestra.run.smithi122.stdout:2187 write 0xe3dcc00 thru 0xe3e4a4a (0x7e4b bytes) 2024-09-19T21:35:11.846 INFO:teuthology.orchestra.run.smithi122.stdout:2189 read 0x5875e00 thru 0x587f222 (0x9423 bytes) 2024-09-19T21:35:11.847 INFO:teuthology.orchestra.run.smithi122.stdout:2190 punch from 0x2d1ba00 to 0x2d1de00, (0x2400 bytes) 2024-09-19T21:35:11.848 INFO:teuthology.orchestra.run.smithi122.stdout:2191 write 0xb1d4600 thru 0xb1e329d (0xec9e bytes) 2024-09-19T21:35:11.852 INFO:teuthology.orchestra.run.smithi122.stdout:2193 read 0x21b8400 thru 0x21b9338 (0xf39 bytes) 2024-09-19T21:35:11.852 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.853 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.853 INFO:teuthology.orchestra.run.smithi122.stdout:2194 write 0x8f28800 thru 0x8f335c9 (0xadca bytes) 2024-09-19T21:35:11.854 INFO:teuthology.orchestra.run.smithi122.stdout:2195 read 0x726c600 thru 0x726ca4d (0x44e bytes) 2024-09-19T21:35:11.855 INFO:teuthology.orchestra.run.smithi122.stdout:2198 read 0xd03c200 thru 0xd03d5aa (0x13ab bytes) 2024-09-19T21:35:11.855 INFO:teuthology.orchestra.run.smithi122.stdout:2201 write 0x23ebc00 thru 0x23efcb7 (0x40b8 bytes) 2024-09-19T21:35:11.857 INFO:teuthology.orchestra.run.smithi122.stdout:2202 punch from 0xb37400 to 0xb46000, (0xec00 bytes) 2024-09-19T21:35:11.858 INFO:teuthology.orchestra.run.smithi122.stdout:2203 read 0x5e89600 thru 0x5e92f29 (0x992a bytes) 2024-09-19T21:35:11.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:11.863 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:11.858+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:11.871 INFO:teuthology.orchestra.run.smithi122.stdout:2204 trunc from 0xe3e4c00 to 0x8f7be00 2024-09-19T21:35:11.900 INFO:teuthology.orchestra.run.smithi122.stdout:2208 read 0x6965200 thru 0x69681a2 (0x2fa3 bytes) 2024-09-19T21:35:11.903 INFO:teuthology.orchestra.run.smithi122.stdout:2209 write 0x7999000 thru 0x79a78e9 (0xe8ea bytes) 2024-09-19T21:35:11.904 INFO:teuthology.orchestra.run.smithi122.stdout:2212 read 0x1cc0600 thru 0x1cc9cb6 (0x96b7 bytes) 2024-09-19T21:35:11.905 INFO:teuthology.orchestra.run.smithi122.stdout:2213 write 0x1b5e200 thru 0x1b5f9ac (0x17ad bytes) 2024-09-19T21:35:11.906 INFO:teuthology.orchestra.run.smithi122.stdout:2214 read 0x5dbc00 thru 0x5df834 (0x3c35 bytes) 2024-09-19T21:35:11.907 INFO:teuthology.orchestra.run.smithi122.stdout:2215 read 0x639b000 thru 0x639c489 (0x148a bytes) 2024-09-19T21:35:11.908 INFO:teuthology.orchestra.run.smithi122.stdout:2216 write 0x78e5c00 thru 0x78f0b32 (0xaf33 bytes) 2024-09-19T21:35:11.909 INFO:teuthology.orchestra.run.smithi122.stdout:2217 read 0x26e1800 thru 0x26edab3 (0xc2b4 bytes) 2024-09-19T21:35:11.910 INFO:teuthology.orchestra.run.smithi122.stdout:2218 read 0x2d7da00 thru 0x2d8a194 (0xc795 bytes) 2024-09-19T21:35:11.926 INFO:teuthology.orchestra.run.smithi122.stdout:2219 trunc from 0x8f7be00 to 0x2668000 2024-09-19T21:35:11.963 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:11.958+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:11.965 INFO:teuthology.orchestra.run.smithi122.stdout:2220 read 0x2136200 thru 0x213b18a (0x4f8b bytes) 2024-09-19T21:35:11.969 INFO:teuthology.orchestra.run.smithi122.stdout:2223 read 0x2424800 thru 0x24329ac (0xe1ad bytes) 2024-09-19T21:35:11.972 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:11.972 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:11.982 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:12.015 INFO:teuthology.orchestra.run.smithi122.stdout:2225 write 0xd502400 thru 0xd50d2be (0xaebf bytes) 2024-09-19T21:35:12.027 INFO:teuthology.orchestra.run.smithi122.stdout:2230 trunc from 0xd50d400 to 0x91fde00 2024-09-19T21:35:12.062 INFO:teuthology.orchestra.run.smithi122.stdout:2231 trunc from 0x91fde00 to 0x82f4200 2024-09-19T21:35:12.064 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:12.062+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:12.097 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.097 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:12.117 INFO:teuthology.orchestra.run.smithi122.stdout:2235 write 0xb946a00 thru 0xb94c3fc (0x59fd bytes) 2024-09-19T21:35:12.119 INFO:teuthology.orchestra.run.smithi122.stdout:2236 write 0x9864a00 thru 0x9866065 (0x1666 bytes) 2024-09-19T21:35:12.120 INFO:teuthology.orchestra.run.smithi122.stdout:2238 clone 15 order 19 su 2048 sc 7 2024-09-19T21:35:12.164 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:12.161+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:12.261 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.261 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.265 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:12.262+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:12.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:12.366 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:12.362+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:12.367 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:35:12.394 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.394 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.404 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:12.467 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:12.462+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:12.508 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone14 from 0xb94c400 (overlap 0x180000) to 0xe6600 2024-09-19T21:35:12.512 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.512 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:12.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:12.568 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:12.566+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:12.643 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.643 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:12.668 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:12.665+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:12.678 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:12.699 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:12.721 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:12.745 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:12.762 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:12.763 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:12.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:12.770 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:12.766+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:12.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:12.793 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:12.795 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:12.796 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:12.871 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:12.866+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:12.881 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:12.881 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:12.891 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:12.972 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:12.970+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1495 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2351104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.602168130000003, 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060406651999999998, 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1495 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400098+0000", 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400097+0000", 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400100+0000", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6249999999999999e-06 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400108+0000", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6070000000000008e-06 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400113+0000", 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1590000000000001e-06 2024-09-19T21:35:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460065+0000", 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05995201 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460302+0000", 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023696000000000001 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460504+0000", 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000201771 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1496 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.602147823000003, 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571117000000001, 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1496 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400118+0000", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400117+0000", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400120+0000", 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4619999999999999e-06 2024-09-19T21:35:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400128+0000", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1070000000000005e-06 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400133+0000", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460527+0000", 2024-09-19T21:35:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060393846000000001 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460605+0000", 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7866000000000003e-05 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460689+0000", 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4345999999999995e-05 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1497 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2367488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.602128360999998, 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060844460000000003, 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1497 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400137+0000", 2024-09-19T21:35:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400139+0000", 2024-09-19T21:35:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4569999999999999e-06 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400159+0000", 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9978000000000001e-05 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400167+0000", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3590000000000002e-06 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460748+0000", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060581191 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460860+0000", 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000112427 2024-09-19T21:35:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.460982+0000", 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121518 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1498 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2375680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.602094049999998, 2024-09-19T21:35:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061004079000000003, 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1498 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400172+0000", 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400171+0000", 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400174+0000", 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5830000000000001e-06 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400186+0000", 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2267e-05 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400192+0000", 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.302e-06 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461007+0000", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060814506999999997 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461088+0000", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1472999999999997e-05 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461176+0000", 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7414e-05 2024-09-19T21:35:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.602068549999998, 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1511 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2482176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.598115321000002, 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060193196999999997, 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1511 2024-09-19T21:35:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404150+0000", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404153+0000", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5840000000000002e-06 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404158+0000", 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6559999999999997e-06 2024-09-19T21:35:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404170+0000", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1402e-05 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464124+0000", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.059954780999999999 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464235+0000", 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011102 2024-09-19T21:35:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464344+0000", 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108216 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1512 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2490368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.598078197, 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060343516, 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1512 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404188+0000", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404187+0000", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404190+0000", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8169999999999999e-06 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404207+0000", 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7462e-05 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404219+0000", 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1338000000000001e-05 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464367+0000", 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060148754999999998 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464450+0000", 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2297000000000002e-05 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464531+0000", 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1306999999999999e-05 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1513 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2498560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.598040978, 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060571107999999999, 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1513 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404225+0000", 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404224+0000", 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404227+0000", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404253+0000", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5638000000000001e-05 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404264+0000", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1253999999999999e-05 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464574+0000", 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060309943999999997 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464677+0000", 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102598 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464796+0000", 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011930900000000001 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1514 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597996588999997, 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060740822, 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1514 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404269+0000", 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404271+0000", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7080000000000002e-06 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404277+0000", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0769999999999996e-06 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404282+0000", 2024-09-19T21:35:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9699999999999998e-06 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464838+0000", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060555789999999998 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.464928+0000", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9382999999999994e-05 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465010+0000", 2024-09-19T21:35:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2368999999999997e-05 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1515 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2514944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597978902000001, 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060967604000000002, 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1515 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404287+0000", 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404286+0000", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404289+0000", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4049999999999998e-06 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404301+0000", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.188e-05 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404306+0000", 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1320000000000002e-06 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465053+0000", 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060747218999999998 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465156+0000", 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102904 2024-09-19T21:35:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465254+0000", 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8521000000000001e-05 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1516 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2523136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597953951999997, 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157074999999998, 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1516 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404312+0000", 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404311+0000", 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404318+0000", 2024-09-19T21:35:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8220000000000001e-06 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404324+0000", 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3199999999999996e-06 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404329+0000", 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465298+0000", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060968899999999999 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465387+0000", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9308000000000006e-05 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465469+0000", 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1397999999999996e-05 2024-09-19T21:35:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597914660999997, 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597877257999997, 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.597859429000003, 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:35:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1533 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2662400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594210650999997, 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060248441999999999, 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1533 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408055+0000", 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408057+0000", 2024-09-19T21:35:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.323e-06 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408067+0000", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.594e-06 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408075+0000", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6500000000000002e-06 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468212+0000", 2024-09-19T21:35:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060136383000000002 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468257+0000", 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5243e-05 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468303+0000", 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6718999999999998e-05 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1534 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2670592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594186106999999, 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060430781000000003, 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1534 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408080+0000", 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408079+0000", 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408082+0000", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408087+0000", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7230000000000001e-06 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408094+0000", 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3749999999999999e-06 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468337+0000", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060243669999999999 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468427+0000", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0006999999999994e-05 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468510+0000", 2024-09-19T21:35:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2905999999999995e-05 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1535 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2678784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594167491999997, 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060516397, 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1535 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408098+0000", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408100+0000", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3750000000000001e-06 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408108+0000", 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3250000000000008e-06 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408117+0000", 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4719999999999995e-06 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468529+0000", 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060412517999999998 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468571+0000", 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1300000000000001e-05 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468615+0000", 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3942e-05 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1536 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594118086000002, 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060670400999999999, 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1536 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408148+0000", 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408147+0000", 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408150+0000", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.892e-06 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408156+0000", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3679999999999998e-06 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408161+0000", 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3059999999999997e-06 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468649+0000", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060487586000000003 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468735+0000", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6040999999999998e-05 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468818+0000", 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2952000000000001e-05 2024-09-19T21:35:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1537 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594099688, 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060787045999999997, 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1537 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408166+0000", 2024-09-19T21:35:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408165+0000", 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408168+0000", 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7530000000000002e-06 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408181+0000", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3209e-05 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408187+0000", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7520000000000002e-06 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468851+0000", 2024-09-19T21:35:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060663918999999997 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468901+0000", 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0244000000000002e-05 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468953+0000", 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1771999999999998e-05 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1538 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.594073919000003, 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060969127999999997, 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1538 2024-09-19T21:35:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408192+0000", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408191+0000", 2024-09-19T21:35:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408194+0000", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.587e-06 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408206+0000", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2364999999999999e-05 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.408211+0000", 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:35:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.468991+0000", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060779300000000001 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469079+0000", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8017999999999997e-05 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.469161+0000", 2024-09-19T21:35:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2157999999999995e-05 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.098 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:13.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:13.100 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:13.093+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:13.200 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:13.197+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:13.223 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-19T21:35:13.254 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:13.254 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:13.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:13.308 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:13.305+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:13.396 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:13.402 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:13.398+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:13.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:13.503 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:13.497+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1075 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 7168~512] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.807044622999999, 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030194739000000002, 2024-09-19T21:35:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1075 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712124+0000", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712126+0000", 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5739999999999998e-06 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712134+0000", 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6550000000000004e-06 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712139+0000", 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2270000000000001e-06 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742210+0000", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030070403999999998 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742253+0000", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3010000000000003e-05 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742319+0000", 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6241000000000005e-05 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1077 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 0~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.807023936, 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030330979000000001, 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1077 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712145+0000", 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712144+0000", 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712148+0000", 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3330000000000001e-06 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712156+0000", 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3280000000000006e-06 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.712161+0000", 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2299999999999999e-06 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742353+0000", 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030191913000000001 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742398+0000", 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4669000000000002e-05 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.742476+0000", 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8181000000000002e-05 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1480 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.786968731000002, 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030120475000000001, 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1480 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732200+0000", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732201+0000", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4160000000000001e-06 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732205+0000", 2024-09-19T21:35:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2169999999999996e-06 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732208+0000", 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6259999999999999e-06 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762084+0000", 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029876117000000001 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762193+0000", 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108906 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762321+0000", 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012741400000000001 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1485 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.786958420000001, 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030359516999999999, 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1485 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732210+0000", 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732212+0000", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5269999999999999e-06 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732215+0000", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.692e-06 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.732218+0000", 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6010000000000002e-06 2024-09-19T21:35:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762393+0000", 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030175415000000001 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762470+0000", 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6774999999999996e-05 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.762570+0000", 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9883000000000006e-05 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1540 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782907080000001, 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.03012836, 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1540 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736262+0000", 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736261+0000", 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736264+0000", 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3120000000000001e-06 2024-09-19T21:35:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736269+0000", 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736274+0000", 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.69e-06 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766159+0000", 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.029884984 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766269+0000", 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000110098 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766390+0000", 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000121489 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1543 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782890082000002, 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030340459, 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1543 2024-09-19T21:35:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736279+0000", 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736278+0000", 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736281+0000", 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4250000000000001e-06 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736286+0000", 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.665e-06 2024-09-19T21:35:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736293+0000", 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5039999999999999e-06 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766437+0000", 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030144388000000001 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766523+0000", 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5575000000000007e-05 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766619+0000", 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6299999999999996e-05 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1546 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782869832999999, 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030580726999999999, 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1546 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736299+0000", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736300+0000", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8369999999999999e-06 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736304+0000", 2024-09-19T21:35:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9890000000000003e-06 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736314+0000", 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2550000000000003e-06 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766650+0000", 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030336609 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766753+0000", 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102413 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766880+0000", 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012705999999999999 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1551 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.78285228, 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030830699999999999, 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1551 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736317+0000", 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736316+0000", 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736322+0000", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8279999999999999e-06 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736325+0000", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.348e-06 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736328+0000", 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7020000000000001e-06 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.766911+0000", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030582668 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767012+0000", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101053 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767147+0000", 2024-09-19T21:35:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000135597 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1554 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782838690999998, 2024-09-19T21:35:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031084074, 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1554 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736330+0000", 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736331+0000", 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5719999999999999e-06 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736334+0000", 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9050000000000001e-06 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736337+0000", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6429999999999999e-06 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767184+0000", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.030846625999999999 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767286+0000", 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010242399999999999 2024-09-19T21:35:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767414+0000", 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012813500000000001 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1559 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782808422999999, 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031279296999999998, 2024-09-19T21:35:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1559 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736360+0000", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736362+0000", 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.362e-06 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736364+0000", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.695e-06 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736367+0000", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767459+0000", 2024-09-19T21:35:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031091534000000001 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767540+0000", 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1353000000000005e-05 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767640+0000", 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9806e-05 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1562 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782798485000001, 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031533528999999998, 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1562 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736370+0000", 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736371+0000", 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.373e-06 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736374+0000", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8480000000000001e-06 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736378+0000", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8850000000000001e-06 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767680+0000", 2024-09-19T21:35:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031301571 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767772+0000", 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2255000000000004e-05 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767904+0000", 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013181299999999999 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1569 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782758084000001, 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031758058999999998, 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1569 2024-09-19T21:35:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736411+0000", 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736410+0000", 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736413+0000", 2024-09-19T21:35:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6960000000000001e-06 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736421+0000", 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6399999999999997e-06 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736425+0000", 2024-09-19T21:35:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5279999999999997e-06 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.767935+0000", 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031509713000000002 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768041+0000", 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106397 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768169+0000", 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000127479 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1571 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782712320999998, 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031967883000000002, 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1571 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736456+0000", 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736459+0000", 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736466+0000", 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5250000000000002e-06 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736470+0000", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6909999999999996e-06 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768196+0000", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031726035 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768298+0000", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010195400000000001 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768424+0000", 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012614899999999999 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1572 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782693956999999, 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032212024999999998, 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1572 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736475+0000", 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736474+0000", 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736477+0000", 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4940000000000002e-06 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736482+0000", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5010000000000004e-06 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736488+0000", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1079999999999998e-06 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768452+0000", 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.031963671999999999 2024-09-19T21:35:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768557+0000", 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104411 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768687+0000", 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000130307 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1574 3.12 3:49a73479:::rbd_data.110682966ae8.0000000000000002:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782676213999999, 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032396917999999997, 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1574 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736493+0000", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736492+0000", 2024-09-19T21:35:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736495+0000", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3640000000000002e-06 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736500+0000", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7069999999999997e-06 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736505+0000", 2024-09-19T21:35:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5959999999999997e-06 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768731+0000", 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032225666 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768798+0000", 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7187000000000006e-05 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768890+0000", 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1730000000000004e-05 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1575 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782658867999999, 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032552508000000001, 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1575 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736510+0000", 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736512+0000", 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736518+0000", 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736523+0000", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5670000000000002e-06 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768914+0000", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032390713000000002 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.768976+0000", 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2390000000000004e-05 2024-09-19T21:35:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769062+0000", 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6248000000000005e-05 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1578 3.12 3:4ff5472a:::rbd_data.110682966ae8.0000000000000008:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782607167999998, 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032774279000000003, 2024-09-19T21:35:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1578 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736562+0000", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736561+0000", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736565+0000", 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.557e-06 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736572+0000", 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9419999999999998e-06 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736576+0000", 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4730000000000002e-06 2024-09-19T21:35:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769102+0000", 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032525674999999997 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769205+0000", 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010330700000000001 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769336+0000", 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013078699999999999 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1579 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782587982999999, 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033004987999999999, 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1579 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736581+0000", 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736580+0000", 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736583+0000", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8090000000000001e-06 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736589+0000", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6570000000000002e-06 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736593+0000", 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5210000000000004e-06 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769367+0000", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032773434999999997 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769463+0000", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6069000000000005e-05 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769586+0000", 2024-09-19T21:35:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012306200000000001 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1582 3.12 3:4f69d4b6:::rbd_data.110682966ae8.0000000000000000:a [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.782506927, 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.033168969999999999, 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1582 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736662+0000", 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736661+0000", 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736663+0000", 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.872e-06 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736668+0000", 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9280000000000001e-06 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.736677+0000", 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2709999999999992e-06 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769651+0000", 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.032974366999999997 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769726+0000", 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5244e-05 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.769831+0000", 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104737 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.770546995, 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:13.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:13.621 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:13.618+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:13.722 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:13.717+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:13.733 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:13.733 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:13.733 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:13.733 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:13.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:13.823 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:13.818+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:13.839 INFO:teuthology.orchestra.run.smithi122.stdout:2239 write 0xc7f4600 thru 0xc7fe030 (0x9a31 bytes) 2024-09-19T21:35:13.840 INFO:teuthology.orchestra.run.smithi122.stdout:2240 read 0xaf04600 thru 0xaf0cba9 (0x85aa bytes) 2024-09-19T21:35:13.846 INFO:teuthology.orchestra.run.smithi122.stdout:2241 read 0x87f1400 thru 0x87f7e94 (0x6a95 bytes) 2024-09-19T21:35:13.850 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:13.850 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:13.851 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:13.851 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:13.851 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:13.858 INFO:teuthology.orchestra.run.smithi122.stdout:2245 trunc from 0xc7fe200 to 0xd222c00 2024-09-19T21:35:13.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:13.908 INFO:teuthology.orchestra.run.smithi122.stdout:2246 write 0x95cd200 thru 0x95db285 (0xe086 bytes) 2024-09-19T21:35:13.909 INFO:teuthology.orchestra.run.smithi122.stdout:2248 read 0xc67ee00 thru 0xc68736e (0x856f bytes) 2024-09-19T21:35:13.910 INFO:teuthology.orchestra.run.smithi122.stdout:2250 read 0x2cfe200 thru 0x2cfe636 (0x437 bytes) 2024-09-19T21:35:13.910 INFO:teuthology.orchestra.run.smithi122.stdout:2253 read 0x7538200 thru 0x75434ac (0xb2ad bytes) 2024-09-19T21:35:13.919 INFO:teuthology.orchestra.run.smithi122.stdout:2255 write 0x54dfc00 thru 0x54e66b7 (0x6ab8 bytes) 2024-09-19T21:35:13.920 INFO:teuthology.orchestra.run.smithi122.stdout:2256 write 0x5cfb600 thru 0x5d03040 (0x7a41 bytes) 2024-09-19T21:35:13.921 INFO:teuthology.orchestra.run.smithi122.stdout:2257 read 0xc50d400 thru 0xc50f315 (0x1f16 bytes) 2024-09-19T21:35:13.922 INFO:teuthology.orchestra.run.smithi122.stdout:2258 read 0x9923000 thru 0x992ffb4 (0xcfb5 bytes) 2024-09-19T21:35:13.923 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:13.918+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:13.930 INFO:teuthology.orchestra.run.smithi122.stdout:2260 read 0x300ca00 thru 0x300d45d (0xa5e bytes) 2024-09-19T21:35:13.936 INFO:teuthology.orchestra.run.smithi122.stdout:2262 trunc from 0xd222c00 to 0xb996000 2024-09-19T21:35:13.989 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4367 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.728534893000003, 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021780411999999999, 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:13.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4367 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257567+0000", 2024-09-19T21:35:13.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257566+0000", 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257569+0000", 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8399999999999999e-06 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257575+0000", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.562e-06 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.257580+0000", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8790000000000002e-06 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.275899+0000", 2024-09-19T21:35:13.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.018319137999999999 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.276109+0000", 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020960099999999999 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279347+0000", 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003237991 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:13.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4414 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.727208423999997, 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021349709000000001, 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4414 2024-09-19T21:35:13.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258893+0000", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258895+0000", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1330000000000002e-06 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258900+0000", 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1410000000000004e-06 2024-09-19T21:35:13.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258904+0000", 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.318e-06 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.279980+0000", 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021076266999999999 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:13.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280104+0000", 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012362199999999999 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280243+0000", 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013863600000000001 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:13.999 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4416 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 485376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.72717695, 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021526086999999999, 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4416 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258924+0000", 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258926+0000", 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2089999999999999e-06 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258932+0000", 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3669999999999996e-06 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258936+0000", 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3030000000000001e-06 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280291+0000", 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021354985999999999 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280379+0000", 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8432999999999998e-05 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280451+0000", 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1109e-05 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4418 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.727137996000003, 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021667024999999999, 2024-09-19T21:35:14.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4418 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:14.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258963+0000", 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258965+0000", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258971+0000", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8529999999999997e-06 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.258982+0000", 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1211e-05 2024-09-19T21:35:14.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280489+0000", 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021506167999999999 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280562+0000", 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3207999999999994e-05 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280630+0000", 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8657000000000004e-05 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4420 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 522240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.727096953, 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021793526000000001, 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4420 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259004+0000", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259012+0000", 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7840000000000004e-06 2024-09-19T21:35:14.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259018+0000", 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6509999999999998e-06 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.259022+0000", 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8729999999999998e-06 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280668+0000", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02164516 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280728+0000", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0346999999999997e-05 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.280798+0000", 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0009e-05 2024-09-19T21:35:14.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4823 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 278528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717844569, 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021199882999999999, 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4823 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:14.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268257+0000", 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268259+0000", 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2409999999999998e-06 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268262+0000", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.695e-06 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268265+0000", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3210000000000001e-06 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289310+0000", 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021044975 2024-09-19T21:35:14.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289388+0000", 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7767000000000003e-05 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289457+0000", 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9093999999999999e-05 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4824 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 288768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717834340000003, 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024097651000000001, 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4824 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.019 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268267+0000", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268268+0000", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.282e-06 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268271+0000", 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7329999999999999e-06 2024-09-19T21:35:14.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268275+0000", 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6710000000000001e-06 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.289482+0000", 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02120791 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290519+0000", 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010363029999999999 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292365+0000", 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018459609999999999 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4825 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.023 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717824898000003, 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022473935, 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4825 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268277+0000", 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268276+0000", 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268278+0000", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.206e-06 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268280+0000", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6199999999999999e-06 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268283+0000", 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:14.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290620+0000", 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022336999999999999 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290692+0000", 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2022999999999998e-05 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290750+0000", 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8668000000000001e-05 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4826 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 325632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717806457999998, 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022588621999999999, 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4826 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268295+0000", 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268297+0000", 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8899999999999999e-06 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268300+0000", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6480000000000001e-06 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268303+0000", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0180000000000002e-06 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290765+0000", 2024-09-19T21:35:14.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022461597999999999 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290802+0000", 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7696e-05 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290884+0000", 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1087000000000005e-05 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4827 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717795850000002, 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022673414999999999, 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4827 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268306+0000", 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268305+0000", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268307+0000", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.508e-06 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268310+0000", 2024-09-19T21:35:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.728e-06 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268312+0000", 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7250000000000002e-06 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290897+0000", 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022585033000000001 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290934+0000", 2024-09-19T21:35:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6942e-05 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290979+0000", 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4688000000000003e-05 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4828 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 362496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717787111, 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022763419, 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4828 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268314+0000", 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268315+0000", 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.311e-06 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268319+0000", 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1549999999999999e-06 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268323+0000", 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1330000000000001e-06 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.290991+0000", 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022668291 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291038+0000", 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7006999999999999e-05 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291078+0000", 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9743999999999998e-05 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4829 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 389120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717774861999999, 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022874156, 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4829 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268327+0000", 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268326+0000", 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268328+0000", 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1890000000000001e-06 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268514+0000", 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185725 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268520+0000", 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6019999999999999e-06 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291090+0000", 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022570277999999999 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291128+0000", 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7623000000000002e-05 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291201+0000", 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3059000000000006e-05 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4830 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 399360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717576047999998, 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022858849000000001, 2024-09-19T21:35:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4830 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268525+0000", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268528+0000", 2024-09-19T21:35:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5540000000000002e-06 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268534+0000", 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.964e-06 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268539+0000", 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8949999999999997e-06 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291222+0000", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.022682455000000001 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291301+0000", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9666000000000006e-05 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291384+0000", 2024-09-19T21:35:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3009000000000004e-05 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4831 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 425984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717558185000001, 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023018005000000001, 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4831 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268543+0000", 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268545+0000", 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.729e-06 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268551+0000", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4480000000000002e-06 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268556+0000", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7770000000000002e-06 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291407+0000", 2024-09-19T21:35:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02285096 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291487+0000", 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0142000000000006e-05 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291561+0000", 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4466000000000002e-05 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4832 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 436224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717541638999997, 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023154310000000001, 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4832 2024-09-19T21:35:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268560+0000", 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268559+0000", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268562+0000", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1519999999999999e-06 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268567+0000", 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268572+0000", 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8670000000000002e-06 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291583+0000", 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02301129 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291653+0000", 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9614000000000001e-05 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291714+0000", 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1222000000000004e-05 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4833 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717525293999998, 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023315366000000001, 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4833 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268576+0000", 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268578+0000", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.429e-06 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268584+0000", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8549999999999998e-06 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268589+0000", 2024-09-19T21:35:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.87e-06 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291737+0000", 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023147607000000001 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291813+0000", 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6243999999999997e-05 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291891+0000", 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8690999999999996e-05 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4834 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717508416999998, 2024-09-19T21:35:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023455496999999999, 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4834 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268593+0000", 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268595+0000", 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.136e-06 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268601+0000", 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7119999999999997e-06 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268607+0000", 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4949999999999996e-06 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291913+0000", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023306459000000002 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.291984+0000", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292048+0000", 2024-09-19T21:35:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4352999999999999e-05 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4835 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717490230000003, 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023586881000000001, 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4835 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268611+0000", 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268613+0000", 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3319999999999999e-06 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268621+0000", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4610000000000002e-06 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268625+0000", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8640000000000004e-06 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292072+0000", 2024-09-19T21:35:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023446132000000001 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292139+0000", 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7228000000000001e-05 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292198+0000", 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9218999999999997e-05 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:4836 3.f 3:f143d14b:::rbd_data.1106b3d15a22.0000000000000002:5 [read 509952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.717471701000001, 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023725810999999999, 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4836 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268630+0000", 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268629+0000", 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268632+0000", 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268638+0000", 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.528e-06 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.268643+0000", 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.011e-06 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292220+0000", 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.023576260000000002 2024-09-19T21:35:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292289+0000", 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9880000000000002e-05 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.292356+0000", 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6088999999999997e-05 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4809.0:1592 3.1d e42/23 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head v 42'106, mlcod=42'104)", 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 18.234318276, 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033307211000000003, 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751783+0000", 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751782+0000", 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758281+0000", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0064992369999999997 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758315+0000", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4561999999999999e-05 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758330+0000", 2024-09-19T21:35:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4822e-05 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758424+0000", 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3377999999999999e-05 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.758462+0000", 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8732000000000001e-05 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785018+0000", 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.026556059999999999 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785090+0000", 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1971999999999995e-05 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.089 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:14.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:14.091 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:14.085+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:14.191 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:14.186+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:14.220 INFO:teuthology.orchestra.run.smithi122.stdout:2263 trunc from 0xb996000 to 0x7d19e00 2024-09-19T21:35:14.254 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:14.254 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:14.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7688 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.234913+0000", 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0088469710000000003, 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0088693269999999998, 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7688 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234913+0000", 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234913+0000", 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234913+0000", 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234917+0000", 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5059999999999997e-06 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234929+0000", 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2094e-05 2024-09-19T21:35:14.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234939+0000", 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5179999999999993e-06 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7720 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235724+0000", 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080362249999999993, 2024-09-19T21:35:14.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0081141819999999993, 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7720 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235724+0000", 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235724+0000", 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235724+0000", 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235726+0000", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4480000000000001e-06 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235733+0000", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5719999999999999e-06 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235737+0000", 2024-09-19T21:35:14.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5639999999999998e-06 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7880 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241867+0000", 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018937240000000001, 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002016538, 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7880 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241867+0000", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241867+0000", 2024-09-19T21:35:14.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241866+0000", 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241868+0000", 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.835e-06 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241872+0000", 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.484e-06 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241875+0000", 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1190000000000001e-06 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7912 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242363+0000", 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013976730000000001, 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0015492119999999999, 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7912 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242363+0000", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242363+0000", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242362+0000", 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242364+0000", 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.497e-06 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242367+0000", 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242369+0000", 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6309999999999999e-06 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7689 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.234948+0000", 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0088119589999999994, 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0089896330000000003, 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7689 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234948+0000", 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234948+0000", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234948+0000", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234952+0000", 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4360000000000004e-06 2024-09-19T21:35:14.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.234990+0000", 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7469000000000003e-05 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235011+0000", 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1749000000000001e-05 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7721 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235741+0000", 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080192260000000008, 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0082298000000000007, 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7721 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.273 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235741+0000", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235741+0000", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235741+0000", 2024-09-19T21:35:14.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235743+0000", 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.385e-06 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235750+0000", 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3719999999999999e-06 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235755+0000", 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4880000000000001e-06 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7881 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241877+0000", 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018833420000000001, 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0021442929999999998, 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7881 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241877+0000", 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241877+0000", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241877+0000", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241878+0000", 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.432e-06 2024-09-19T21:35:14.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241881+0000", 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8669999999999998e-06 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241884+0000", 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.65e-06 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7913 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242371+0000", 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001388823, 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001700827, 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7913 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242371+0000", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242371+0000", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242371+0000", 2024-09-19T21:35:14.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242373+0000", 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.519e-06 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242376+0000", 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.785e-06 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242378+0000", 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5950000000000001e-06 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7690 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235023+0000", 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0087377329999999993, 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0091017600000000004, 2024-09-19T21:35:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7690 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235023+0000", 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235023+0000", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235022+0000", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235028+0000", 2024-09-19T21:35:14.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0619999999999997e-06 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235041+0000", 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2942000000000001e-05 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235050+0000", 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6879999999999994e-06 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7722 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235759+0000", 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080015180000000009, 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0084196180000000002, 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7722 2024-09-19T21:35:14.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235759+0000", 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235759+0000", 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235758+0000", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235761+0000", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3460000000000001e-06 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235766+0000", 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2079999999999999e-06 2024-09-19T21:35:14.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235771+0000", 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4700000000000001e-06 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7882 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241886+0000", 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018743620000000001, 2024-09-19T21:35:14.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002343033, 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7882 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241886+0000", 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241886+0000", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241886+0000", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241887+0000", 2024-09-19T21:35:14.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.584e-06 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241890+0000", 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.717e-06 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241892+0000", 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4899999999999999e-06 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7914 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242380+0000", 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013799769999999999, 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018999690000000001, 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7914 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.294 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242380+0000", 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242380+0000", 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242380+0000", 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242391+0000", 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0651e-05 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242394+0000", 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242397+0000", 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7691 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235058+0000", 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0087020780000000002, 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0092742970000000008, 2024-09-19T21:35:14.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7691 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235058+0000", 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235058+0000", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235058+0000", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235062+0000", 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2019999999999997e-06 2024-09-19T21:35:14.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235073+0000", 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1616e-05 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235082+0000", 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6260000000000001e-06 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7723 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235775+0000", 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0079850379999999999, 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0085873760000000007, 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7723 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235775+0000", 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235775+0000", 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235775+0000", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235777+0000", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3779999999999999e-06 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235783+0000", 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1580000000000002e-06 2024-09-19T21:35:14.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235788+0000", 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4680000000000003e-06 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7883 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241894+0000", 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001865804, 2024-09-19T21:35:14.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024938569999999999, 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7883 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241894+0000", 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241894+0000", 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241894+0000", 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241896+0000", 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.606e-06 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241899+0000", 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7410000000000001e-06 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241901+0000", 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.61e-06 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.307 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7915 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242400+0000", 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013606860000000001, 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002026538, 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7915 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242400+0000", 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242400+0000", 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242399+0000", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242401+0000", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.548e-06 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242404+0000", 2024-09-19T21:35:14.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.768e-06 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242407+0000", 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4819999999999999e-06 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7692 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235089+0000", 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0086708979999999998, 2024-09-19T21:35:14.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0093757930000000003, 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7692 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235089+0000", 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235089+0000", 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235089+0000", 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235093+0000", 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1320000000000004e-06 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235103+0000", 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6560000000000004e-06 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235111+0000", 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0069999999999997e-06 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.314 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7724 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235793+0000", 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0079674749999999999, 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0086987709999999992, 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7724 2024-09-19T21:35:14.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235793+0000", 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235793+0000", 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235792+0000", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235795+0000", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3020000000000002e-06 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235800+0000", 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0839999999999999e-06 2024-09-19T21:35:14.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235804+0000", 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4299999999999999e-06 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7884 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241903+0000", 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018569630000000001, 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026176229999999999, 2024-09-19T21:35:14.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7884 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241903+0000", 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241903+0000", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241903+0000", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241905+0000", 2024-09-19T21:35:14.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4020000000000001e-06 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241907+0000", 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7259999999999998e-06 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241910+0000", 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5129999999999999e-06 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7916 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242409+0000", 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013510099999999999, 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0021368870000000001, 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7916 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242409+0000", 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242409+0000", 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242409+0000", 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242410+0000", 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3829999999999999e-06 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242413+0000", 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6589999999999999e-06 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242416+0000", 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5579999999999999e-06 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7693 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235119+0000", 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0086417109999999998, 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0094551170000000007, 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7693 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235119+0000", 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235119+0000", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235118+0000", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235123+0000", 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7079999999999996e-06 2024-09-19T21:35:14.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235155+0000", 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2709000000000001e-05 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235163+0000", 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7919999999999993e-06 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7725 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235808+0000", 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0079523509999999999, 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0087914940000000004, 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7725 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235808+0000", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235808+0000", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235808+0000", 2024-09-19T21:35:14.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235810+0000", 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2699999999999999e-06 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235815+0000", 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0300000000000001e-06 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235820+0000", 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7779999999999998e-06 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7885 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241913+0000", 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018474419999999999, 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0027141420000000001, 2024-09-19T21:35:14.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7885 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241913+0000", 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241913+0000", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241913+0000", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241914+0000", 2024-09-19T21:35:14.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.387e-06 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241917+0000", 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9229999999999998e-06 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241921+0000", 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8759999999999998e-06 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7917 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242418+0000", 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013426320000000001, 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0022589060000000002, 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7917 2024-09-19T21:35:14.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242418+0000", 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242418+0000", 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242417+0000", 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242419+0000", 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6169999999999999e-06 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242422+0000", 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7240000000000001e-06 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242424+0000", 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6230000000000001e-06 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7694 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235169+0000", 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0085917059999999993, 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0095590829999999995, 2024-09-19T21:35:14.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7694 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235169+0000", 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235169+0000", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235168+0000", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235171+0000", 2024-09-19T21:35:14.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2540000000000001e-06 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235189+0000", 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.713e-05 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235194+0000", 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9220000000000002e-06 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.342 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7726 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235825+0000", 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0079354630000000002, 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0089560839999999996, 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7726 2024-09-19T21:35:14.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235825+0000", 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235825+0000", 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235824+0000", 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235827+0000", 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.334e-06 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235832+0000", 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1839999999999998e-06 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235836+0000", 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7886 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241923+0000", 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018373199999999999, 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002909146, 2024-09-19T21:35:14.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7886 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241923+0000", 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241923+0000", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241923+0000", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241924+0000", 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5230000000000001e-06 2024-09-19T21:35:14.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241928+0000", 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8550000000000004e-06 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241931+0000", 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8140000000000001e-06 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7918 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242427+0000", 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001333713, 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024530649999999999, 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7918 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.350 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242427+0000", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242427+0000", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242426+0000", 2024-09-19T21:35:14.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242428+0000", 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.362e-06 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242430+0000", 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6450000000000001e-06 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242433+0000", 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.875e-06 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7695 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235199+0000", 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0085617160000000005, 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0097184379999999994, 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7695 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235199+0000", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235199+0000", 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235198+0000", 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235207+0000", 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5429999999999994e-06 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235213+0000", 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1480000000000003e-06 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235218+0000", 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8060000000000002e-06 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7727 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235840+0000", 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0079200820000000002, 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0091032830000000002, 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7727 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235840+0000", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235840+0000", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235840+0000", 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235842+0000", 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4499999999999998e-06 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235847+0000", 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1359999999999996e-06 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235852+0000", 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.724e-06 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7887 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241933+0000", 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018273899999999999, 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030362259999999999, 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7887 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241933+0000", 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241933+0000", 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241933+0000", 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241934+0000", 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3030000000000001e-06 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241937+0000", 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8899999999999999e-06 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241939+0000", 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4770000000000002e-06 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7919 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242435+0000", 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013249589999999999, 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025590980000000001, 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7919 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242435+0000", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242435+0000", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242435+0000", 2024-09-19T21:35:14.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242436+0000", 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.384e-06 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242439+0000", 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6350000000000002e-06 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242442+0000", 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7696 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235270+0000", 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084907400000000001, 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0097666220000000008, 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7696 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235270+0000", 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235270+0000", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235269+0000", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235272+0000", 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.394e-06 2024-09-19T21:35:14.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235281+0000", 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.276e-06 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235286+0000", 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5199999999999997e-06 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7728 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235873+0000", 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0078877340000000004, 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0092150860000000008, 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7728 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.371 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235873+0000", 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235873+0000", 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235872+0000", 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235877+0000", 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9300000000000002e-06 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235888+0000", 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1807000000000001e-05 2024-09-19T21:35:14.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235898+0000", 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2220000000000008e-06 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7888 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241947+0000", 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018133229999999999, 2024-09-19T21:35:14.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003192051, 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7888 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241947+0000", 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241947+0000", 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241947+0000", 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241949+0000", 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.024e-06 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241952+0000", 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.467e-06 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241955+0000", 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7949999999999999e-06 2024-09-19T21:35:14.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7920 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242444+0000", 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0013164730000000001, 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0028018819999999999, 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7920 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242444+0000", 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242444+0000", 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242444+0000", 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242445+0000", 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.468e-06 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242448+0000", 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.661e-06 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242450+0000", 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6170000000000001e-06 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.381 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7697 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235290+0000", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084698559999999996, 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010007957, 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7697 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235290+0000", 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235290+0000", 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235290+0000", 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235293+0000", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7860000000000001e-06 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235302+0000", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7030000000000001e-06 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235307+0000", 2024-09-19T21:35:14.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6380000000000003e-06 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7729 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235906+0000", 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0078541610000000001, 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0094475129999999994, 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.385 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7729 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235906+0000", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235906+0000", 2024-09-19T21:35:14.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235905+0000", 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235910+0000", 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4299999999999999e-06 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235921+0000", 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1026000000000001e-05 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235929+0000", 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3599999999999996e-06 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7889 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241957+0000", 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0018030570000000001, 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034400640000000001, 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7889 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241957+0000", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241957+0000", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241957+0000", 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241958+0000", 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3510000000000001e-06 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241961+0000", 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8430000000000001e-06 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241964+0000", 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5730000000000002e-06 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7921 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242453+0000", 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001307711, 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029732740000000001, 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7921 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242453+0000", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242453+0000", 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242452+0000", 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242454+0000", 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3820000000000001e-06 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242457+0000", 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8200000000000001e-06 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242459+0000", 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7180000000000001e-06 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7698 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235311+0000", 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084491470000000006, 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010153291, 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7698 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.396 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235311+0000", 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235311+0000", 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235311+0000", 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235313+0000", 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6639999999999998e-06 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235319+0000", 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2619999999999998e-06 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235323+0000", 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6e-06 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7730 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235938+0000", 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0078219859999999995, 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0095789629999999994, 2024-09-19T21:35:14.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7730 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235938+0000", 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235938+0000", 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235938+0000", 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235942+0000", 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7919999999999999e-06 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235954+0000", 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1952e-05 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235962+0000", 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1219999999999995e-06 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.402 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7890 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241967+0000", 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001792994, 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003604922, 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7890 2024-09-19T21:35:14.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241967+0000", 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241967+0000", 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241967+0000", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241969+0000", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.474e-06 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241971+0000", 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9220000000000001e-06 2024-09-19T21:35:14.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241974+0000", 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4959999999999999e-06 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7922 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242461+0000", 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001298942, 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031620139999999999, 2024-09-19T21:35:14.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7922 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242461+0000", 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242461+0000", 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242461+0000", 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242463+0000", 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3710000000000001e-06 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242465+0000", 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6749999999999998e-06 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242468+0000", 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6759999999999999e-06 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.409 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7699 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235327+0000", 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084332130000000002, 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010350817999999999, 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7699 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235327+0000", 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235327+0000", 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235327+0000", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235329+0000", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4559999999999999e-06 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235334+0000", 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.118e-06 2024-09-19T21:35:14.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235339+0000", 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3599999999999998e-06 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7731 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235970+0000", 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0077903460000000001, 2024-09-19T21:35:14.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0097347630000000004, 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7731 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235970+0000", 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235970+0000", 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235969+0000", 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235974+0000", 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5580000000000002e-06 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235995+0000", 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1586999999999998e-05 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236001+0000", 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8119999999999996e-06 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.416 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7891 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241977+0000", 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001783764, 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003756419, 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7891 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241977+0000", 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241977+0000", 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241976+0000", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241978+0000", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4419999999999999e-06 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241980+0000", 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7379999999999999e-06 2024-09-19T21:35:14.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241983+0000", 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5890000000000001e-06 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7923 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242471+0000", 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0012890519999999999, 2024-09-19T21:35:14.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0032869470000000001, 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7923 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242471+0000", 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242471+0000", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242471+0000", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242472+0000", 2024-09-19T21:35:14.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.46e-06 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242475+0000", 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6709999999999999e-06 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242478+0000", 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5900000000000002e-06 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7700 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235342+0000", 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084178289999999999, 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010444790000000001, 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7700 2024-09-19T21:35:14.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235342+0000", 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235342+0000", 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235342+0000", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235345+0000", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5299999999999999e-06 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235350+0000", 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1290000000000004e-06 2024-09-19T21:35:14.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235354+0000", 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6859999999999997e-06 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7732 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.236006+0000", 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0077539810000000001, 2024-09-19T21:35:14.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0098073589999999999, 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7732 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236006+0000", 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236006+0000", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236006+0000", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236014+0000", 2024-09-19T21:35:14.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4940000000000003e-06 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236020+0000", 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1360000000000002e-06 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236025+0000", 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1379999999999998e-06 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7892 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241985+0000", 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0017750279999999999, 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0038572820000000001, 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7892 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.431 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241985+0000", 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241985+0000", 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241985+0000", 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241986+0000", 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.347e-06 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241989+0000", 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6589999999999999e-06 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241992+0000", 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7924 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242480+0000", 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00128038, 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033981969999999999, 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7924 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242480+0000", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242480+0000", 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242480+0000", 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242488+0000", 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3329999999999997e-06 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242494+0000", 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6910000000000002e-06 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242496+0000", 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7669999999999999e-06 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7701 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235358+0000", 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0084021700000000005, 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010575403000000001, 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7701 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.438 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235358+0000", 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235358+0000", 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235358+0000", 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235360+0000", 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.402e-06 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235367+0000", 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3119999999999996e-06 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235372+0000", 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7500000000000003e-06 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7733 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.236029+0000", 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0077307890000000001, 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0099635479999999992, 2024-09-19T21:35:14.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7733 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236029+0000", 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236029+0000", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236029+0000", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236032+0000", 2024-09-19T21:35:14.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.582e-06 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236037+0000", 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1490000000000002e-06 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236041+0000", 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6360000000000002e-06 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7893 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.241995+0000", 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0017651260000000001, 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0040499189999999999, 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7893 2024-09-19T21:35:14.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241995+0000", 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241995+0000", 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241995+0000", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241996+0000", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.285e-06 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.241999+0000", 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:14.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242003+0000", 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1269999999999996e-06 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7925 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242499+0000", 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00126129, 2024-09-19T21:35:14.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003591173, 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7925 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242499+0000", 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242499+0000", 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242499+0000", 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242500+0000", 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.685e-06 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242503+0000", 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9220000000000001e-06 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242506+0000", 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7719999999999999e-06 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7702 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235377+0000", 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083833039999999994, 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010742807, 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7702 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235377+0000", 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235377+0000", 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235377+0000", 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235379+0000", 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4820000000000001e-06 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235384+0000", 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0980000000000001e-06 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235389+0000", 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4499999999999997e-06 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7734 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.236045+0000", 2024-09-19T21:35:14.455 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0077151199999999998, 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010104597, 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7734 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236045+0000", 2024-09-19T21:35:14.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236045+0000", 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236045+0000", 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236047+0000", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.43e-06 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236052+0000", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0180000000000001e-06 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236057+0000", 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.656e-06 2024-09-19T21:35:14.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7894 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242009+0000", 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0017510889999999999, 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0041668269999999997, 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7894 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242009+0000", 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242009+0000", 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242009+0000", 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242011+0000", 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242018+0000", 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8789999999999997e-06 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242023+0000", 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7690000000000004e-06 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7926 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242508+0000", 2024-09-19T21:35:14.462 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001252167, 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0037096569999999999, 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7926 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242508+0000", 2024-09-19T21:35:14.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242508+0000", 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242508+0000", 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242520+0000", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2302e-05 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242527+0000", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3820000000000001e-06 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242530+0000", 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3739999999999998e-06 2024-09-19T21:35:14.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7703 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235392+0000", 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083678539999999992, 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010881205999999999, 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7703 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235392+0000", 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235392+0000", 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235392+0000", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235394+0000", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4679999999999999e-06 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235400+0000", 2024-09-19T21:35:14.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0869999999999997e-06 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235404+0000", 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5639999999999998e-06 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7735 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.236061+0000", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0076995680000000004, 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.010269102, 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7735 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236061+0000", 2024-09-19T21:35:14.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236061+0000", 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236060+0000", 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236063+0000", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236068+0000", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2120000000000002e-06 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.236072+0000", 2024-09-19T21:35:14.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5480000000000003e-06 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7895 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242027+0000", 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001733152, 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0043552460000000001, 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7895 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242027+0000", 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242027+0000", 2024-09-19T21:35:14.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242027+0000", 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242030+0000", 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.796e-06 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242039+0000", 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4550000000000001e-06 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242042+0000", 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.476 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7927 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242564+0000", 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001196527, 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0038544870000000002, 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7927 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242564+0000", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242564+0000", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242563+0000", 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242566+0000", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9100000000000001e-06 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242573+0000", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3799999999999999e-06 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242578+0000", 2024-09-19T21:35:14.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0490000000000003e-06 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7704 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235408+0000", 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083523130000000001, 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011039393, 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7704 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235408+0000", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235408+0000", 2024-09-19T21:35:14.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235408+0000", 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235410+0000", 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4700000000000001e-06 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235417+0000", 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3860000000000001e-06 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235422+0000", 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4519999999999999e-06 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7896 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242053+0000", 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0017071409999999999, 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0044255090000000002, 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7896 2024-09-19T21:35:14.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242053+0000", 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242053+0000", 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242053+0000", 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242056+0000", 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.066e-06 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242073+0000", 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6866000000000001e-05 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242078+0000", 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3170000000000001e-06 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7928 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242582+0000", 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0011785210000000001, 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039595619999999998, 2024-09-19T21:35:14.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7928 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242582+0000", 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242582+0000", 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242581+0000", 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242584+0000", 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.255e-06 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242590+0000", 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242594+0000", 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7705 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235426+0000", 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083344790000000005, 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011172952999999999, 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7705 2024-09-19T21:35:14.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235426+0000", 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235426+0000", 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235425+0000", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235428+0000", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4839999999999998e-06 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235434+0000", 2024-09-19T21:35:14.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1299999999999998e-06 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235439+0000", 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7897 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242099+0000", 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0016610279999999999, 2024-09-19T21:35:14.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0045541999999999996, 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7897 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242099+0000", 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242099+0000", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242099+0000", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242102+0000", 2024-09-19T21:35:14.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4510000000000001e-06 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242109+0000", 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.08e-06 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242115+0000", 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.356e-06 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7929 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242598+0000", 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0011620370000000001, 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0040970750000000004, 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7929 2024-09-19T21:35:14.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242598+0000", 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242598+0000", 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242598+0000", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242600+0000", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3300000000000001e-06 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242606+0000", 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:14.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242612+0000", 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7989999999999999e-06 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7706 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235442+0000", 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083178829999999999, 2024-09-19T21:35:14.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011337748999999999, 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7706 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235442+0000", 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235442+0000", 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235442+0000", 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235444+0000", 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.486e-06 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235450+0000", 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1630000000000004e-06 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235454+0000", 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5120000000000002e-06 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.504 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7898 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242119+0000", 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001641417, 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0046993369999999996, 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7898 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242119+0000", 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242119+0000", 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242119+0000", 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242121+0000", 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242127+0000", 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.784e-06 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242133+0000", 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4450000000000001e-06 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7930 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242632+0000", 2024-09-19T21:35:14.508 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0011284019999999999, 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0042238789999999998, 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7930 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242632+0000", 2024-09-19T21:35:14.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242632+0000", 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242631+0000", 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242634+0000", 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5239999999999999e-06 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242641+0000", 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1500000000000002e-06 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242647+0000", 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.429e-06 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.511 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7707 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235458+0000", 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0083023460000000004, 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011436017, 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7707 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235458+0000", 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235458+0000", 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235458+0000", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235464+0000", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.844e-06 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235470+0000", 2024-09-19T21:35:14.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5899999999999998e-06 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235474+0000", 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5140000000000003e-06 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7899 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242137+0000", 2024-09-19T21:35:14.515 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0016228270000000001, 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0048095730000000001, 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7899 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242137+0000", 2024-09-19T21:35:14.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242137+0000", 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242137+0000", 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242140+0000", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5100000000000001e-06 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242146+0000", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242150+0000", 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4000000000000002e-06 2024-09-19T21:35:14.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7931 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242651+0000", 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001109206, 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0043468350000000003, 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7931 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242651+0000", 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242651+0000", 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242651+0000", 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242653+0000", 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8090000000000001e-06 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242658+0000", 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0270000000000003e-06 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242665+0000", 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6579999999999996e-06 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7708 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235479+0000", 2024-09-19T21:35:14.522 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0082816780000000007, 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011574795000000001, 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7708 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235479+0000", 2024-09-19T21:35:14.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235479+0000", 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235478+0000", 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235481+0000", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4399999999999999e-06 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235486+0000", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235490+0000", 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3819999999999997e-06 2024-09-19T21:35:14.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7900 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242154+0000", 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001606339, 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0049499879999999998, 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7900 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242154+0000", 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242154+0000", 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242154+0000", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242156+0000", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4380000000000002e-06 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242162+0000", 2024-09-19T21:35:14.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6969999999999998e-06 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242166+0000", 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.583e-06 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7932 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242670+0000", 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001089837, 2024-09-19T21:35:14.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0044858249999999997, 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7932 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242670+0000", 2024-09-19T21:35:14.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242670+0000", 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242670+0000", 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242673+0000", 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5620000000000002e-06 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242678+0000", 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4179999999999996e-06 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242683+0000", 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8149999999999996e-06 2024-09-19T21:35:14.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7709 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235529+0000", 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0082309470000000006, 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011684607, 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7709 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235529+0000", 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235529+0000", 2024-09-19T21:35:14.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235529+0000", 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235531+0000", 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235537+0000", 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8200000000000002e-06 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235542+0000", 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6840000000000004e-06 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7901 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.536 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242170+0000", 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0015900669999999999, 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050959109999999998, 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7901 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242170+0000", 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242170+0000", 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242170+0000", 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242172+0000", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.469e-06 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242178+0000", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.468e-06 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242183+0000", 2024-09-19T21:35:14.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7350000000000004e-06 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7933 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242687+0000", 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001073524, 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0046339399999999996, 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7933 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242687+0000", 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242687+0000", 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242686+0000", 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242689+0000", 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3130000000000001e-06 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242694+0000", 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242699+0000", 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4689999999999999e-06 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.543 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7710 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235546+0000", 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0082146579999999997, 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011827549999999999, 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7710 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235546+0000", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235546+0000", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235545+0000", 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235548+0000", 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3980000000000002e-06 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235553+0000", 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1660000000000002e-06 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235557+0000", 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3379999999999998e-06 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7902 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242198+0000", 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0015620110000000001, 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052297749999999999, 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7902 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242198+0000", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242198+0000", 2024-09-19T21:35:14.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242198+0000", 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242201+0000", 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5920000000000001e-06 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242208+0000", 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6409999999999996e-06 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242215+0000", 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7390000000000002e-06 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.550 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7934 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242703+0000", 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0010574180000000001, 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0047776049999999999, 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7934 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242703+0000", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242703+0000", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242702+0000", 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242705+0000", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2139999999999999e-06 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242710+0000", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7350000000000001e-06 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242715+0000", 2024-09-19T21:35:14.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4000000000000002e-06 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7711 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235561+0000", 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0081992620000000006, 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011972488999999999, 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7711 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235561+0000", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235561+0000", 2024-09-19T21:35:14.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235561+0000", 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235563+0000", 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2979999999999999e-06 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235568+0000", 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.152e-06 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235573+0000", 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.724e-06 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.557 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7903 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242219+0000", 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0015409620000000001, 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0053659170000000004, 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7903 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242219+0000", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242219+0000", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242219+0000", 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242222+0000", 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7879999999999998e-06 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242228+0000", 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9939999999999997e-06 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242231+0000", 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.191e-06 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7935 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242723+0000", 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001037224, 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0049157180000000003, 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7935 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242723+0000", 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242723+0000", 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242723+0000", 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242725+0000", 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6170000000000001e-06 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242731+0000", 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6579999999999999e-06 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242736+0000", 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7419999999999997e-06 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7712 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235578+0000", 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0081820079999999993, 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012112275, 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7712 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235578+0000", 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235578+0000", 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235578+0000", 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235580+0000", 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.582e-06 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235586+0000", 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2120000000000002e-06 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235592+0000", 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6619999999999999e-06 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7904 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242251+0000", 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0015092529999999999, 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0054922529999999999, 2024-09-19T21:35:14.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7904 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242251+0000", 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242251+0000", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242251+0000", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242253+0000", 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2900000000000001e-06 2024-09-19T21:35:14.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242257+0000", 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4240000000000003e-06 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242262+0000", 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2409999999999997e-06 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7936 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242773+0000", 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.000987246, 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050156970000000004, 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7936 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242773+0000", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242773+0000", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242773+0000", 2024-09-19T21:35:14.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242775+0000", 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2570000000000002e-06 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242780+0000", 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9119999999999997e-06 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242783+0000", 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4249999999999998e-06 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7713 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235596+0000", 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0081640989999999993, 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012219600000000001, 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7713 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235596+0000", 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235596+0000", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235596+0000", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235598+0000", 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4339999999999999e-06 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235605+0000", 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4080000000000003e-06 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235609+0000", 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3839999999999999e-06 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7905 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242264+0000", 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001496331, 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0055804799999999996, 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7905 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242264+0000", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242264+0000", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242264+0000", 2024-09-19T21:35:14.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242265+0000", 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6249999999999999e-06 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242268+0000", 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8310000000000001e-06 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242271+0000", 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7999999999999999e-06 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7937 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242787+0000", 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.000973743, 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050853729999999998, 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7937 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242787+0000", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242787+0000", 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242786+0000", 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242788+0000", 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.73e-06 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242791+0000", 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.875e-06 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242794+0000", 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6220000000000001e-06 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7714 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235613+0000", 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0081474890000000008, 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012302820000000001, 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7714 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235613+0000", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235613+0000", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235612+0000", 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235615+0000", 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2019999999999998e-06 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235620+0000", 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1819999999999997e-06 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235624+0000", 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4479999999999996e-06 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7906 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242273+0000", 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001487185, 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056933390000000004, 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7906 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242273+0000", 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242273+0000", 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242273+0000", 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242274+0000", 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.683e-06 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242278+0000", 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5260000000000002e-06 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242282+0000", 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1370000000000004e-06 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7938 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242796+0000", 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00096440299999999996, 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0052256890000000004, 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7938 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242796+0000", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242796+0000", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242796+0000", 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242797+0000", 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.624e-06 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242800+0000", 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.819e-06 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242803+0000", 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6220000000000001e-06 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7715 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235628+0000", 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0081322809999999999, 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012444154000000001, 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.596 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7715 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235628+0000", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235628+0000", 2024-09-19T21:35:14.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235628+0000", 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235630+0000", 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.278e-06 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235635+0000", 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.074e-06 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235639+0000", 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2420000000000002e-06 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.599 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7907 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242286+0000", 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001474153, 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0058272640000000004, 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7907 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242286+0000", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242286+0000", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242286+0000", 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242288+0000", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.39e-06 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242294+0000", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5799999999999999e-06 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242298+0000", 2024-09-19T21:35:14.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4719999999999997e-06 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:8122 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.245043+0000", 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.9987173, 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030968279999999998, 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 8122 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245043+0000", 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245043+0000", 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245042+0000", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245048+0000", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.241e-06 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245079+0000", 2024-09-19T21:35:14.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0219e-05 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245087+0000", 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3100000000000001e-06 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7716 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.606 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235643+0000", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.008117038, 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012524540000000001, 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7716 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235643+0000", 2024-09-19T21:35:14.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235643+0000", 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235643+0000", 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235645+0000", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5969999999999999e-06 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235650+0000", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1100000000000002e-06 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235655+0000", 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3379999999999998e-06 2024-09-19T21:35:14.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7908 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242302+0000", 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001458158, 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0059102349999999998, 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7908 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242302+0000", 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242302+0000", 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242302+0000", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242305+0000", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8839999999999998e-06 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242314+0000", 2024-09-19T21:35:14.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4909999999999994e-06 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242317+0000", 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1580000000000001e-06 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:8123 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.245101+0000", 2024-09-19T21:35:14.613 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.9986596, 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031476059999999998, 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 8123 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245101+0000", 2024-09-19T21:35:14.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245101+0000", 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245100+0000", 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245104+0000", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1609999999999996e-06 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245111+0000", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2649999999999999e-06 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245117+0000", 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9909999999999999e-06 2024-09-19T21:35:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7717 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235672+0000", 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080883220000000002, 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012626551999999999, 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7717 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235672+0000", 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235672+0000", 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235671+0000", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235674+0000", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5739999999999998e-06 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235679+0000", 2024-09-19T21:35:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5380000000000002e-06 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235684+0000", 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6600000000000003e-06 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7909 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242320+0000", 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001440254, 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0060312639999999997, 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7909 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242320+0000", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242320+0000", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242320+0000", 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242322+0000", 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.792e-06 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242324+0000", 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9170000000000002e-06 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242327+0000", 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6759999999999999e-06 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:8124 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.245125+0000", 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.9986353, 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0032819540000000001, 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 8124 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245125+0000", 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245125+0000", 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245124+0000", 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245134+0000", 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5470000000000002e-06 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245141+0000", 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4490000000000002e-06 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245147+0000", 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4299999999999997e-06 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7718 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235688+0000", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080722720000000001, 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012762264000000001, 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7718 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235688+0000", 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235688+0000", 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235688+0000", 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235690+0000", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3980000000000002e-06 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235695+0000", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0889999999999999e-06 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235699+0000", 2024-09-19T21:35:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.42e-06 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7910 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242331+0000", 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001429442, 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0061489830000000002, 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7910 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242331+0000", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242331+0000", 2024-09-19T21:35:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242331+0000", 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242332+0000", 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242335+0000", 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7640000000000001e-06 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242338+0000", 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5919999999999999e-06 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:8125 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.245160+0000", 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.998601, 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033465420000000001, 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 8125 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245160+0000", 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245160+0000", 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245159+0000", 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245162+0000", 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3780000000000001e-06 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245169+0000", 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8469999999999998e-06 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245182+0000", 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2772000000000001e-05 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7719 3.1c 3.dbf7541c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.235703+0000", 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0080568589999999995, 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012830172000000001, 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7719 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235703+0000", 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235703+0000", 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235703+0000", 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235705+0000", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2740000000000002e-06 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235710+0000", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0459999999999996e-06 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.235715+0000", 2024-09-19T21:35:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4360000000000004e-06 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:7911 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.242352+0000", 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001408416, 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0062077260000000002, 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 7911 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242352+0000", 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242352+0000", 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242352+0000", 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242354+0000", 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0650000000000001e-06 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242357+0000", 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6330000000000002e-06 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.242360+0000", 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9679999999999998e-06 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:8126 3.d 3.ecad7ead (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.245189+0000", 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4294967295.9985709, 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034077259999999998, 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 8126 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245189+0000", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245189+0000", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245189+0000", 2024-09-19T21:35:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245211+0000", 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1997e-05 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245226+0000", 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5400000000000002e-05 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.245234+0000", 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6969999999999993e-06 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 112 2024-09-19T21:35:14.649 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:14.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2265 punch from 0x6901800 to 0x690ec00, (0xd400 bytes) 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2266 punch from 0x587a400 to 0x5888400, (0xe000 bytes) 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2267 write 0x6b93e00 thru 0x6b9448b (0x68c bytes) 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2269 read 0x2359800 thru 0x235cdfb (0x35fc bytes) 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2270 read 0x550ea00 thru 0x550f04d (0x64e bytes) 2024-09-19T21:35:14.652 INFO:teuthology.orchestra.run.smithi122.stdout:2273 trunc from 0x7d19e00 to 0x3547a00 2024-09-19T21:35:14.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:14.653 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:14.646+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:14.743 INFO:teuthology.orchestra.run.smithi122.stdout:2274 read 0x300be00 thru 0x300cee9 (0x10ea bytes) 2024-09-19T21:35:14.751 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:14.750+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:14.776 INFO:teuthology.orchestra.run.smithi122.stdout:2275 write 0x53f4200 thru 0x53f9620 (0x5421 bytes) 2024-09-19T21:35:14.777 INFO:teuthology.orchestra.run.smithi122.stdout:2276 read 0x4c78000 thru 0x4c78961 (0x962 bytes) 2024-09-19T21:35:14.777 INFO:teuthology.orchestra.run.smithi122.stdout:2277 write 0x5c9200 thru 0x5d1d5b (0x8b5c bytes) 2024-09-19T21:35:14.778 INFO:teuthology.orchestra.run.smithi122.stdout:2278 read 0x52a200 thru 0x5314d4 (0x72d5 bytes) 2024-09-19T21:35:14.785 INFO:teuthology.orchestra.run.smithi122.stdout:2280 write 0x4d4f400 thru 0x4d56599 (0x719a bytes) 2024-09-19T21:35:14.785 INFO:teuthology.orchestra.run.smithi122.stdout:2281 read 0x2d49a00 thru 0x2d553b8 (0xb9b9 bytes) 2024-09-19T21:35:14.794 INFO:teuthology.orchestra.run.smithi122.stdout:2282 read 0x4aa0a00 thru 0x4aad36c (0xc96d bytes) 2024-09-19T21:35:14.804 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:14.804 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:14.804 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:14.804 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:14.804 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:14.807 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:14.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:14.821 INFO:teuthology.orchestra.run.smithi122.stdout:2285 trunc from 0x53f9800 to 0xe906200 2024-09-19T21:35:14.861 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:14.858+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:14.954 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:14.950+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:14.989 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.052842103000003, 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:14.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:14.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:14.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:14.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:14.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15154 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5314560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.343214615, 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056594848000000003, 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15154 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:14.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642372+0000", 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999456 2024-09-19T21:35:14.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642431+0000", 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8517000000000002e-05 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642439+0000", 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1980000000000001e-06 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:14.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642445+0000", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3980000000000004e-06 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698816+0000", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056370908999999997 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698937+0000", 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000120976 2024-09-19T21:35:15.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699021+0000", 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4206e-05 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15155 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5328896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34304928699999998, 2024-09-19T21:35:15.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056657103, 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15155 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:15.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642450+0000", 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998584 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642601+0000", 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015027000000000001 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642618+0000", 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7482999999999998e-05 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642623+0000", 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9409999999999998e-06 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699053+0000", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056430165999999997 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699162+0000", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108626 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699249+0000", 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7064999999999994e-05 2024-09-19T21:35:15.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15156 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5343232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34292788899999999, 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056741714999999998, 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15156 2024-09-19T21:35:15.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642628+0000", 2024-09-19T21:35:15.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999146 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642716+0000", 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8770000000000006e-05 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642722+0000", 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6860000000000003e-06 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642726+0000", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9489999999999998e-06 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699283+0000", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556510999999997 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699378+0000", 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5079000000000002e-05 2024-09-19T21:35:15.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699455+0000", 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7245999999999999e-05 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15157 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5357568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34264683499999998, 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056672473000000001, 2024-09-19T21:35:15.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15157 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642729+0000", 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997349 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642999+0000", 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026962200000000001 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643008+0000", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6589999999999996e-06 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643014+0000", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.888e-06 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699488+0000", 2024-09-19T21:35:15.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056473795 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699590+0000", 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010186100000000001 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699667+0000", 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6627999999999999e-05 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15158 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5371904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34240509800000002, 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056666994999999998, 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15158 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643020+0000", 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997845 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643241+0000", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022081999999999999 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643248+0000", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0389999999999999e-06 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643255+0000", 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7059999999999998e-06 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699701+0000", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056446488000000003 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699820+0000", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119053 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699903+0000", 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2595000000000005e-05 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.018 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15159 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5386240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34220948299999998, 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056686874999999998, 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15159 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.019 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643259+0000", 2024-09-19T21:35:15.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998279 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643435+0000", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017585800000000001 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643442+0000", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7889999999999997e-06 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643446+0000", 2024-09-19T21:35:15.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4710000000000001e-06 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699931+0000", 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056484178000000003 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700036+0000", 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010582 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700119+0000", 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2052999999999995e-05 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15160 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5400576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34210759299999999, 2024-09-19T21:35:15.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056812712000000001, 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15160 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643450+0000", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999166 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643536+0000", 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6331999999999999e-05 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643541+0000", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5900000000000001e-06 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643545+0000", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9049999999999999e-06 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700149+0000", 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056604067000000001 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700262+0000", 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113408 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700346+0000", 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3808999999999997e-05 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.027 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15161 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5414912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.341913091, 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056828916, 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15161 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643548+0000", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998202 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643731+0000", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018323300000000001 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643737+0000", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.088e-06 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643742+0000", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.284e-06 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700374+0000", 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056632688 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700481+0000", 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106531 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700557+0000", 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5995999999999995e-05 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.031 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15162 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5429248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34179283500000002, 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056920353, 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15162 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.032 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643745+0000", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999897 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643852+0000", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106959 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643862+0000", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9529999999999994e-06 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643869+0000", 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9199999999999998e-06 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700585+0000", 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056716492 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700685+0000", 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700769+0000", 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3264999999999996e-05 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15163 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5443584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.341284278, 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056623993999999997, 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15163 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:15.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643873+0000", 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9995165 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644360+0000", 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048695100000000001 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644370+0000", 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6809999999999993e-06 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644374+0000", 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.639e-06 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700825+0000", 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056450137999999997 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700915+0000", 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0080000000000005e-05 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700981+0000", 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6202000000000001e-05 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15164 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5457920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.040 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34126263299999998, 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056756384, 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15164 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644380+0000", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9640000000000002e-06 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644384+0000", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644388+0000", 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5040000000000004e-06 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701006+0000", 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056617546999999997 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701079+0000", 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3755000000000001e-05 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701135+0000", 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448999999999997e-05 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15165 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5472256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34124888199999998, 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056848226000000002, 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15165 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644395+0000", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9919999999999999e-06 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644401+0000", 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2310000000000001e-06 2024-09-19T21:35:15.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644406+0000", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1429999999999997e-06 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701150+0000", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056743891999999997 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701201+0000", 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0614e-05 2024-09-19T21:35:15.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701240+0000", 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9697000000000003e-05 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15166 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5486592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.341124337, 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056825742999999998, 2024-09-19T21:35:15.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15166 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644410+0000", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998937 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644521+0000", 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011017400000000001 2024-09-19T21:35:15.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644526+0000", 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644530+0000", 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701255+0000", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056724519000000001 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701292+0000", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7425000000000002e-05 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701343+0000", 2024-09-19T21:35:15.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0553999999999997e-05 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15167 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5500928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34089976399999999, 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056707884, 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15167 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644545+0000", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998035 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644746+0000", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200779 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644756+0000", 2024-09-19T21:35:15.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0916e-05 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644761+0000", 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.228e-06 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701357+0000", 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056595897999999999 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701412+0000", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5634000000000003e-05 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701449+0000", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7017999999999997e-05 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15168 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5515264~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:15.058 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.340657653, 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056638200999999999, 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15168 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:15.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644775+0000", 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997921 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644987+0000", 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021153200000000001 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644999+0000", 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2089999999999999e-05 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645004+0000", 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4020000000000004e-06 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701471+0000", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056467824999999999 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701560+0000", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8195999999999994e-05 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701622+0000", 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2130000000000003e-05 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.062 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15170 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5543936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.340278305, 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056639429999999998, 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15170 2024-09-19T21:35:15.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645362+0000", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645365+0000", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0779999999999998e-06 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645368+0000", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6720000000000002e-06 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645371+0000", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2480000000000001e-06 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701839+0000", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056468061 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701937+0000", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7979999999999994e-05 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702002+0000", 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4770000000000005e-05 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15171 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5558272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.34000243499999999, 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556991000000001, 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15171 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645374+0000", 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997354 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645642+0000", 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026817799999999999 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645648+0000", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4389999999999997e-06 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645653+0000", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1949999999999996e-06 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702027+0000", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056374806 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702126+0000", 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8941e-05 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702196+0000", 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9239999999999994e-05 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15172 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5572608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.33980780599999999, 2024-09-19T21:35:15.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056523239000000003, 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15172 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645656+0000", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998231 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645837+0000", 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018102 2024-09-19T21:35:15.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645845+0000", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0649999999999999e-06 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645850+0000", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6619999999999996e-06 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702226+0000", 2024-09-19T21:35:15.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056376060999999998 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702303+0000", 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7055e-05 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702357+0000", 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3517999999999998e-05 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15173 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5586944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.33967847800000001, 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056512898999999998, 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15173 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645853+0000", 2024-09-19T21:35:15.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998908 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645966+0000", 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011243600000000001 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645972+0000", 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.066e-06 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645977+0000", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9320000000000004e-06 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702371+0000", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394702999999997 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702428+0000", 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6508000000000002e-05 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702476+0000", 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7576e-05 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.080 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:15.081 INFO:teuthology.orchestra.run.smithi122.stdout:2288 punch from 0x24c00 to 0x2b600, (0x6a00 bytes) 2024-09-19T21:35:15.081 INFO:teuthology.orchestra.run.smithi122.stdout:2290 trunc from 0xe906200 to 0xdce0c00 2024-09-19T21:35:15.082 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:15.083 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:15.083 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:15.083 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:15.077+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:15.089 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:15.131 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:15.155 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:15.157 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:15.158 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:15.212 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:15.210+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.218 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:15.275 INFO:teuthology.orchestra.run.smithi122.stdout:2291 write 0x9f41800 thru 0x9f50182 (0xe983 bytes) 2024-09-19T21:35:15.277 INFO:teuthology.orchestra.run.smithi122.stdout:2295 write 0xc0f5000 thru 0xc0feaf5 (0x9af6 bytes) 2024-09-19T21:35:15.277 INFO:teuthology.orchestra.run.smithi122.stdout:2296 write 0xe95400 thru 0xe988cc (0x34cd bytes) 2024-09-19T21:35:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:2297 write 0x7f73000 thru 0x7f7fa77 (0xca78 bytes) 2024-09-19T21:35:15.279 INFO:teuthology.orchestra.run.smithi122.stdout:2300 read 0x7f2e00 thru 0x7f9c09 (0x6e0a bytes) 2024-09-19T21:35:15.283 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:15.278+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:15.285 INFO:teuthology.orchestra.run.smithi122.stdout:2302 write 0x18fb400 thru 0x18fe21f (0x2e20 bytes) 2024-09-19T21:35:15.308 INFO:teuthology.orchestra.run.smithi122.stdout:2307 trunc from 0xdce0c00 to 0x4458800 2024-09-19T21:35:15.334 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.335 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:15.383 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:15.377+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:15.457 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:15.457 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:15.458 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T21:35:15.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:35:15.484 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:15.481+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:15.511 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.511 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:15.585 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:15.581+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:15.602 INFO:teuthology.orchestra.run.smithi122.stdout:2308 punch from 0x2c47e00 to 0x2c55000, (0xd200 bytes) 2024-09-19T21:35:15.650 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.650 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:15.686 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:15.681+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:15.742 INFO:teuthology.orchestra.run.smithi122.stdout:2310 write 0xb865200 thru 0xb871548 (0xc349 bytes) 2024-09-19T21:35:15.743 INFO:teuthology.orchestra.run.smithi122.stdout:2311 write 0x16a9600 thru 0x16ab4a1 (0x1ea2 bytes) 2024-09-19T21:35:15.745 INFO:teuthology.orchestra.run.smithi122.stdout:2312 read 0xb06fa00 thru 0xb071b10 (0x2111 bytes) 2024-09-19T21:35:15.745 INFO:teuthology.orchestra.run.smithi122.stdout:2314 read 0x6066e00 thru 0x607439e (0xd59f bytes) 2024-09-19T21:35:15.758 INFO:teuthology.orchestra.run.smithi122.stdout:2315 trunc from 0xb871600 to 0x5f0bc00 2024-09-19T21:35:15.784 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.784 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.787 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:15.781+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:15.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:15.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:15.862 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:15.862 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 52, 2024-09-19T21:35:15.862 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:35:12.346616+0000", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:34:04.640393+0000", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5804032, 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "nodeep-scrub", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:35:15.863 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "jewel", 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:35:15.864 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:15.865 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:15.866 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:15.867 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:15.868 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:15.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:35:15.870 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:15.871 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:15.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:35:15.873 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:15.874 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:15.875 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:15.876 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.877 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.25, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.25, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:15.881 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:15.882 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:15.883 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "52", 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:15.884 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 18, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 52, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:15.885 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:15.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:15.887 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.1299999952316284, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.1299999952316284, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-19T21:35:15.888 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:15.889 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.890 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.891 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.893 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.895 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.896 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:35:15.900 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:35:15.901 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:15.902 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.904 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.906 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:15.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.908 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:35:15.909 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 24, 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:35:15.910 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.911 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:35:15.913 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:35:15.914 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.915 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:35:15.916 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.917 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.920 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:15.922 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.923 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.924 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.928 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.929 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.930 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.930 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.930 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:35:15.930 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.931 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:15.931 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.931 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.931 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.932 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.932 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.932 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:35:15.932 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.934 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:15.934 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.934 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.934 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.935 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:15.936 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:35:15.937 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:35:15.938 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:35:15.939 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:35:15.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:15.939 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:2318 read 0x4a39200 thru 0x4a47761 (0xe562 bytes) 2024-09-19T21:35:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:2319 read 0x3737800 thru 0x373e457 (0x6c58 bytes) 2024-09-19T21:35:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:2320 write 0x936a00 thru 0x945819 (0xee1a bytes) 2024-09-19T21:35:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:2322 read 0xe6d000 thru 0xe70d1a (0x3d1b bytes) 2024-09-19T21:35:15.941 INFO:teuthology.orchestra.run.smithi122.stdout:2323 read 0x4696600 thru 0x469a1d1 (0x3bd2 bytes) 2024-09-19T21:35:15.942 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:35:12.346616+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:34:04.640393+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 52, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:35:15.942 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-19T21:35:15.943 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:35:15.943 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:15.938+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:15.966 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:15.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:16.040 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:16.037+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:16.086 INFO:teuthology.orchestra.run.smithi122.stdout:2324 write 0x7286200 thru 0x7296160 (0xff61 bytes) 2024-09-19T21:35:16.087 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.087 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.088 INFO:teuthology.orchestra.run.smithi122.stdout:2325 read 0x297e000 thru 0x29810c2 (0x30c3 bytes) 2024-09-19T21:35:16.091 INFO:teuthology.orchestra.run.smithi122.stdout:2328 read 0x62ef200 thru 0x62efbb8 (0x9b9 bytes) 2024-09-19T21:35:16.093 INFO:teuthology.orchestra.run.smithi122.stdout:2332 trunc from 0x7296200 to 0x6e34000 2024-09-19T21:35:16.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:16.142 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:16.138+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:16.148 INFO:teuthology.orchestra.run.smithi122.stdout:2333 read 0x3feba00 thru 0x3ff52dd (0x98de bytes) 2024-09-19T21:35:16.186 INFO:teuthology.orchestra.run.smithi122.stdout:2336 write 0xdb7a800 thru 0xdb84016 (0x9817 bytes) 2024-09-19T21:35:16.199 INFO:teuthology.orchestra.run.smithi122.stdout:2338 trunc from 0xdb84200 to 0x804ac00 2024-09-19T21:35:16.207 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.207 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:16.243 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:16.237+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:16.246 INFO:teuthology.orchestra.run.smithi122.stdout:2339 read 0x527f200 thru 0x5283f07 (0x4d08 bytes) 2024-09-19T21:35:16.249 INFO:teuthology.orchestra.run.smithi122.stdout:2341 write 0x3807000 thru 0x380855d (0x155e bytes) 2024-09-19T21:35:16.249 INFO:teuthology.orchestra.run.smithi122.stdout:2343 write 0x355c400 thru 0x3569543 (0xd144 bytes) 2024-09-19T21:35:16.250 INFO:teuthology.orchestra.run.smithi122.stdout:2344 write 0x5474c00 thru 0x548295a (0xdd5b bytes) 2024-09-19T21:35:16.250 INFO:teuthology.orchestra.run.smithi122.stdout:2345 read 0x11f4400 thru 0x11fbfb9 (0x7bba bytes) 2024-09-19T21:35:16.255 INFO:teuthology.orchestra.run.smithi122.stdout:2346 read 0x6bc6800 thru 0x6bd03fc (0x9bfd bytes) 2024-09-19T21:35:16.256 INFO:teuthology.orchestra.run.smithi122.stdout:2347 punch from 0x7a87e00 to 0x7a92c00, (0xae00 bytes) 2024-09-19T21:35:16.260 INFO:teuthology.orchestra.run.smithi122.stdout:2349 trunc from 0x804ac00 to 0x73e3c00 2024-09-19T21:35:16.323 INFO:teuthology.orchestra.run.smithi122.stdout:2351 trunc from 0x73e3c00 to 0x3270000 2024-09-19T21:35:16.329 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.329 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:16.385 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:16.381+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:16.446 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:16.441+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:16.500 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.500 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:16.547 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:16.541+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:16.563 INFO:teuthology.orchestra.run.smithi122.stdout:2352 read 0xa50e00 thru 0xa5f832 (0xea33 bytes) 2024-09-19T21:35:16.566 INFO:teuthology.orchestra.run.smithi122.stdout:2353 read 0x2c29000 thru 0x2c2990e (0x90f bytes) 2024-09-19T21:35:16.567 INFO:teuthology.orchestra.run.smithi122.stdout:2355 punch from 0x5d0600 to 0x5d7000, (0x6a00 bytes) 2024-09-19T21:35:16.567 INFO:teuthology.orchestra.run.smithi122.stdout:2356 read 0x18a00 thru 0x1d5bc (0x4bbd bytes) 2024-09-19T21:35:16.571 INFO:teuthology.orchestra.run.smithi122.stdout:2357 read 0x8ff600 thru 0x906018 (0x6a19 bytes) 2024-09-19T21:35:16.573 INFO:teuthology.orchestra.run.smithi122.stdout:2358 trunc from 0x3270000 to 0x311cc00 2024-09-19T21:35:16.591 INFO:teuthology.orchestra.run.smithi122.stdout:2361 read 0x257800 thru 0x263f4c (0xc74d bytes) 2024-09-19T21:35:16.622 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.622 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.622 INFO:teuthology.orchestra.run.smithi122.stdout:2363 write 0x58a4200 thru 0x58abed7 (0x7cd8 bytes) 2024-09-19T21:35:16.624 INFO:teuthology.orchestra.run.smithi122.stdout:2364 write 0x1a82a00 thru 0x1a91f44 (0xf545 bytes) 2024-09-19T21:35:16.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:16.663 INFO:teuthology.orchestra.run.smithi122.stdout:2365 write 0x6626200 thru 0x662da30 (0x7831 bytes) 2024-09-19T21:35:16.664 INFO:teuthology.orchestra.run.smithi122.stdout:2366 punch from 0x265200 to 0x26ac00, (0x5a00 bytes) 2024-09-19T21:35:16.667 INFO:teuthology.orchestra.run.smithi122.stdout:2368 trunc from 0x662dc00 to 0x6a5a200 2024-09-19T21:35:16.677 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:16.673+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:16.701 INFO:teuthology.orchestra.run.smithi122.stdout:2369 read 0x2785400 thru 0x2793e3e (0xea3f bytes) 2024-09-19T21:35:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:2370 trunc from 0x6a5a200 to 0x31d7c00 2024-09-19T21:35:16.749 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:16.745+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:16.786 INFO:teuthology.orchestra.run.smithi122.stdout:2373 trunc from 0x31d7c00 to 0xd252000 2024-09-19T21:35:16.795 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.795 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:16.806 INFO:teuthology.orchestra.run.smithi122.stdout:2374 write 0x2932600 thru 0x29419d5 (0xf3d6 bytes) 2024-09-19T21:35:16.808 INFO:teuthology.orchestra.run.smithi122.stdout:2375 punch from 0xc1a8200 to 0xc1b0200, (0x8000 bytes) 2024-09-19T21:35:16.852 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:16.850+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:16.885 INFO:teuthology.orchestra.run.smithi122.stdout:2377 read 0x8c44e00 thru 0x8c517df (0xc9e0 bytes) 2024-09-19T21:35:16.885 INFO:teuthology.orchestra.run.smithi122.stdout:2379 write 0x6d9ac00 thru 0x6da45f4 (0x99f5 bytes) 2024-09-19T21:35:16.885 INFO:teuthology.orchestra.run.smithi122.stdout:2380 write 0x8b1fc00 thru 0x8b2e0c0 (0xe4c1 bytes) 2024-09-19T21:35:16.905 INFO:teuthology.orchestra.run.smithi122.stdout:2381 trunc from 0xd252000 to 0x35d2e00 2024-09-19T21:35:16.951 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:16.946+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:16.963 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:16.964 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:16.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:17.052 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:17.050+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:17.096 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:17.096 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:17.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:17.107 INFO:teuthology.orchestra.run.smithi122.stdout:2384 punch from 0xfc9800 to 0xfd1000, (0x7800 bytes) 2024-09-19T21:35:17.133 INFO:teuthology.orchestra.run.smithi122.stdout:2385 write 0x54dc000 thru 0x54e77cf (0xb7d0 bytes) 2024-09-19T21:35:17.153 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:17.150+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:17.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:17.188 INFO:teuthology.orchestra.run.smithi122.stdout:2386 write 0xd232a00 thru 0xd23cd99 (0xa39a bytes) 2024-09-19T21:35:17.191 INFO:teuthology.orchestra.run.smithi122.stdout:2390 punch from 0x7899e00 to 0x78a1a00, (0x7c00 bytes) 2024-09-19T21:35:17.192 INFO:teuthology.orchestra.run.smithi122.stdout:2393 punch from 0x12b8400 to 0x12bd800, (0x5400 bytes) 2024-09-19T21:35:17.253 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:17.253 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:17.255 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:17.249+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:17.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:17.267 INFO:teuthology.orchestra.run.smithi122.stdout:2394 trunc from 0xd23ce00 to 0x584a200 2024-09-19T21:35:17.355 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:17.349+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:17.364 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:17.381 INFO:teuthology.orchestra.run.smithi122.stdout:2395 punch from 0x19d8e00 to 0x19da800, (0x1a00 bytes) 2024-09-19T21:35:17.383 INFO:teuthology.orchestra.run.smithi122.stdout:2396 read 0x12a0e00 thru 0x12a18ec (0xaed bytes) 2024-09-19T21:35:17.386 INFO:teuthology.orchestra.run.smithi122.stdout:2397 write 0x4b36800 thru 0x4b3893d (0x213e bytes) 2024-09-19T21:35:17.386 INFO:teuthology.orchestra.run.smithi122.stdout:2398 read 0x3883200 thru 0x388c1f4 (0x8ff5 bytes) 2024-09-19T21:35:17.386 INFO:teuthology.orchestra.run.smithi122.stdout:2402 write 0x464e000 thru 0x465af7d (0xcf7e bytes) 2024-09-19T21:35:17.388 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:17.389 INFO:teuthology.orchestra.run.smithi122.stdout:2405 trunc from 0x584a200 to 0x4d54000 2024-09-19T21:35:17.410 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:17.411 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:17.411 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:17.411 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:17.416 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:17.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:17.439 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:17.456 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:17.449+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:17.457 INFO:teuthology.orchestra.run.smithi122.stdout:2407 write 0x596d000 thru 0x5970801 (0x3802 bytes) 2024-09-19T21:35:17.458 INFO:teuthology.orchestra.run.smithi122.stdout:2408 punch from 0xfa7000 to 0xfafa00, (0x8a00 bytes) 2024-09-19T21:35:17.459 INFO:teuthology.orchestra.run.smithi122.stdout:2410 punch from 0x34c5000 to 0x34c7e00, (0x2e00 bytes) 2024-09-19T21:35:17.460 INFO:teuthology.orchestra.run.smithi122.stdout:2413 read 0x1cac200 thru 0x1cbb13d (0xef3e bytes) 2024-09-19T21:35:17.469 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:17.494 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:17.496 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:17.497 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:17.502 INFO:teuthology.orchestra.run.smithi122.stdout:2414 write 0xc69dc00 thru 0xc6a6e2f (0x9230 bytes) 2024-09-19T21:35:17.503 INFO:teuthology.orchestra.run.smithi122.stdout:2415 punch from 0x83f4e00 to 0x8402a00, (0xdc00 bytes) 2024-09-19T21:35:17.505 INFO:teuthology.orchestra.run.smithi122.stdout:2416 write 0x7968600 thru 0x797375b (0xb15c bytes) 2024-09-19T21:35:17.514 INFO:teuthology.orchestra.run.smithi122.stdout:2418 trunc from 0xc6a7000 to 0xabbde00 2024-09-19T21:35:17.541 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:17.541 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:17.541 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:17.551 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:17.556 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:17.553+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:17.569 INFO:teuthology.orchestra.run.smithi122.stdout:2419 punch from 0x2468600 to 0x246ae00, (0x2800 bytes) 2024-09-19T21:35:17.611 INFO:teuthology.orchestra.run.smithi122.stdout:2420 read 0x9b11600 thru 0x9b2135f (0xfd60 bytes) 2024-09-19T21:35:17.612 INFO:teuthology.orchestra.run.smithi122.stdout:2422 write 0x18aac00 thru 0x18b35bd (0x89be bytes) 2024-09-19T21:35:17.612 INFO:teuthology.orchestra.run.smithi122.stdout:2424 read 0x4872c00 thru 0x487eb58 (0xbf59 bytes) 2024-09-19T21:35:17.657 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:17.653+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:17.696 INFO:teuthology.orchestra.run.smithi122.stdout:2427 trunc from 0xabbde00 to 0x70f4a00 2024-09-19T21:35:17.709 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.773113889000001, 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:17.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:17.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:17.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15154 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5314560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.063486401, 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056594848000000003, 2024-09-19T21:35:17.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15154 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642372+0000", 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999456 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642431+0000", 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8517000000000002e-05 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642439+0000", 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1980000000000001e-06 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642445+0000", 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3980000000000004e-06 2024-09-19T21:35:17.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698816+0000", 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056370908999999997 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698937+0000", 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000120976 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699021+0000", 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4206e-05 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15155 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5328896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.063321073, 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056657103, 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15155 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642450+0000", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998584 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642601+0000", 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015027000000000001 2024-09-19T21:35:17.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642618+0000", 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7482999999999998e-05 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642623+0000", 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9409999999999998e-06 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699053+0000", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056430165999999997 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699162+0000", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108626 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699249+0000", 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7064999999999994e-05 2024-09-19T21:35:17.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15156 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5343232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0631996749999999, 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056741714999999998, 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15156 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642628+0000", 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999146 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642716+0000", 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8770000000000006e-05 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642722+0000", 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6860000000000003e-06 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642726+0000", 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9489999999999998e-06 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699283+0000", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556510999999997 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699378+0000", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5079000000000002e-05 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699455+0000", 2024-09-19T21:35:17.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7245999999999999e-05 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15157 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5357568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0629186210000001, 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056672473000000001, 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15157 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642729+0000", 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997349 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642999+0000", 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026962200000000001 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643008+0000", 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6589999999999996e-06 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643014+0000", 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.888e-06 2024-09-19T21:35:17.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699488+0000", 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056473795 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699590+0000", 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010186100000000001 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699667+0000", 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6627999999999999e-05 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15158 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5371904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.062676884, 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056666994999999998, 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15158 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643020+0000", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997845 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643241+0000", 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022081999999999999 2024-09-19T21:35:17.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643248+0000", 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0389999999999999e-06 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643255+0000", 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7059999999999998e-06 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699701+0000", 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056446488000000003 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699820+0000", 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119053 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699903+0000", 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2595000000000005e-05 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15159 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5386240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0624812690000001, 2024-09-19T21:35:17.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056686874999999998, 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15159 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:17.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643259+0000", 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998279 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643435+0000", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017585800000000001 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643442+0000", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7889999999999997e-06 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643446+0000", 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4710000000000001e-06 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699931+0000", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056484178000000003 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700036+0000", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010582 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700119+0000", 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2052999999999995e-05 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15160 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5400576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0623793789999998, 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056812712000000001, 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15160 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643450+0000", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999166 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643536+0000", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6331999999999999e-05 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643541+0000", 2024-09-19T21:35:17.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5900000000000001e-06 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643545+0000", 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9049999999999999e-06 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700149+0000", 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056604067000000001 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700262+0000", 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113408 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700346+0000", 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3808999999999997e-05 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15161 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5414912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.062184877, 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056828916, 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15161 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643548+0000", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998202 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643731+0000", 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018323300000000001 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643737+0000", 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.088e-06 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643742+0000", 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.284e-06 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700374+0000", 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056632688 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700481+0000", 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106531 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700557+0000", 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5995999999999995e-05 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15162 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5429248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0620646210000002, 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056920353, 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15162 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643745+0000", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999897 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643852+0000", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106959 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643862+0000", 2024-09-19T21:35:17.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9529999999999994e-06 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643869+0000", 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9199999999999998e-06 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700585+0000", 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056716492 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700685+0000", 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700769+0000", 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3264999999999996e-05 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.761 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15163 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5443584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0615560639999999, 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056623993999999997, 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15163 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:17.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643873+0000", 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9995165 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644360+0000", 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048695100000000001 2024-09-19T21:35:17.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644370+0000", 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6809999999999993e-06 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644374+0000", 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.639e-06 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700825+0000", 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056450137999999997 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700915+0000", 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0080000000000005e-05 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700981+0000", 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6202000000000001e-05 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15164 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5457920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.061534419, 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056756384, 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15164 2024-09-19T21:35:17.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644380+0000", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9640000000000002e-06 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644384+0000", 2024-09-19T21:35:17.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644388+0000", 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5040000000000004e-06 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701006+0000", 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056617546999999997 2024-09-19T21:35:17.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701079+0000", 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3755000000000001e-05 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701135+0000", 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448999999999997e-05 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.771 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15165 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5472256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.061520668, 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056848226000000002, 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15165 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644395+0000", 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9919999999999999e-06 2024-09-19T21:35:17.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644401+0000", 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2310000000000001e-06 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644406+0000", 2024-09-19T21:35:17.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1429999999999997e-06 2024-09-19T21:35:17.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701150+0000", 2024-09-19T21:35:17.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056743891999999997 2024-09-19T21:35:17.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701201+0000", 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0614e-05 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701240+0000", 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9697000000000003e-05 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15166 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5486592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0613961230000002, 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056825742999999998, 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15166 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644410+0000", 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998937 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644521+0000", 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011017400000000001 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644526+0000", 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644530+0000", 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701255+0000", 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056724519000000001 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701292+0000", 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7425000000000002e-05 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701343+0000", 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0553999999999997e-05 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15167 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5500928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0611715500000001, 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056707884, 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15167 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644545+0000", 2024-09-19T21:35:17.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998035 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644746+0000", 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200779 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644756+0000", 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0916e-05 2024-09-19T21:35:17.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644761+0000", 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.228e-06 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701357+0000", 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056595897999999999 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701412+0000", 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5634000000000003e-05 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701449+0000", 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7017999999999997e-05 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15168 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5515264~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0609294390000001, 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056638200999999999, 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.790 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.790 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.790 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15168 2024-09-19T21:35:17.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644775+0000", 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997921 2024-09-19T21:35:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644987+0000", 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021153200000000001 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644999+0000", 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2089999999999999e-05 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645004+0000", 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4020000000000004e-06 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701471+0000", 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056467824999999999 2024-09-19T21:35:17.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701560+0000", 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8195999999999994e-05 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701622+0000", 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2130000000000003e-05 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15170 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5543936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:17.797 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0605500910000001, 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056639429999999998, 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15170 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:17.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645362+0000", 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645365+0000", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0779999999999998e-06 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645368+0000", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6720000000000002e-06 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645371+0000", 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2480000000000001e-06 2024-09-19T21:35:17.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701839+0000", 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056468061 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701937+0000", 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7979999999999994e-05 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702002+0000", 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4770000000000005e-05 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15171 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5558272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0602742209999998, 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556991000000001, 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15171 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645374+0000", 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997354 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645642+0000", 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026817799999999999 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645648+0000", 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4389999999999997e-06 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645653+0000", 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1949999999999996e-06 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702027+0000", 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056374806 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702126+0000", 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8941e-05 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702196+0000", 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9239999999999994e-05 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.806 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15172 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5572608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0600795920000001, 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056523239000000003, 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15172 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645656+0000", 2024-09-19T21:35:17.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998231 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645837+0000", 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018102 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645845+0000", 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0649999999999999e-06 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645850+0000", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6619999999999996e-06 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702226+0000", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056376060999999998 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702303+0000", 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7055e-05 2024-09-19T21:35:17.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702357+0000", 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3517999999999998e-05 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15173 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5586944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:17.811 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0599502639999998, 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056512898999999998, 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15173 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:17.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645853+0000", 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998908 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645966+0000", 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011243600000000001 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645972+0000", 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.066e-06 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645977+0000", 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9320000000000004e-06 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702371+0000", 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394702999999997 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702428+0000", 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6508000000000002e-05 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702476+0000", 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7576e-05 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.816 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:17.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:17.819 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:17.814+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2428 read 0x4350000 thru 0x4359110 (0x9111 bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2431 write 0xa0fd600 thru 0xa1032d9 (0x5cda bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2433 punch from 0x82b8c00 to 0x82b9800, (0xc00 bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2434 read 0x88d4600 thru 0x88d5f2e (0x192f bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2435 write 0x2781a00 thru 0x27839e0 (0x1fe1 bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2436 read 0x49c2800 thru 0x49c5e3b (0x363c bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2438 read 0x6563400 thru 0x6572233 (0xee34 bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2440 read 0x557600 thru 0x5660b3 (0xeab4 bytes) 2024-09-19T21:35:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2441 write 0xe3e2e00 thru 0xe3ef32d (0xc52e bytes) 2024-09-19T21:35:17.821 INFO:teuthology.orchestra.run.smithi122.stdout:2445 read 0x8823600 thru 0x883058b (0xcf8c bytes) 2024-09-19T21:35:17.821 INFO:teuthology.orchestra.run.smithi122.stdout:2447 read 0x1cdc000 thru 0x1ce0596 (0x4597 bytes) 2024-09-19T21:35:17.821 INFO:teuthology.orchestra.run.smithi122.stdout:2450 read 0x7729c00 thru 0x77309be (0x6dbf bytes) 2024-09-19T21:35:17.821 INFO:teuthology.orchestra.run.smithi122.stdout:2451 punch from 0x368cc00 to 0x369b000, (0xe400 bytes) 2024-09-19T21:35:17.823 INFO:teuthology.orchestra.run.smithi122.stdout:2452 read 0x5da9800 thru 0x5db3e8b (0xa68c bytes) 2024-09-19T21:35:17.823 INFO:teuthology.orchestra.run.smithi122.stdout:2453 write 0x4aec400 thru 0x4af8f8f (0xcb90 bytes) 2024-09-19T21:35:17.823 INFO:teuthology.orchestra.run.smithi122.stdout:2455 write 0x2bfcc00 thru 0x2c0781e (0xac1f bytes) 2024-09-19T21:35:17.857 INFO:teuthology.orchestra.run.smithi122.stdout:2456 write 0x4cda400 thru 0x4cdf821 (0x5422 bytes) 2024-09-19T21:35:17.858 INFO:teuthology.orchestra.run.smithi122.stdout:2457 read 0x6cd4c00 thru 0x6ce0752 (0xbb53 bytes) 2024-09-19T21:35:17.859 INFO:teuthology.orchestra.run.smithi122.stdout:2458 read 0x740600 thru 0x74d634 (0xd035 bytes) 2024-09-19T21:35:17.861 INFO:teuthology.orchestra.run.smithi122.stdout:2461 read 0x81d0c00 thru 0x81d5a7e (0x4e7f bytes) 2024-09-19T21:35:17.863 INFO:teuthology.orchestra.run.smithi122.stdout:2463 read 0x8c9ca00 thru 0x8ca74e3 (0xaae4 bytes) 2024-09-19T21:35:17.863 INFO:teuthology.orchestra.run.smithi122.stdout:2465 read 0x5082a00 thru 0x5090a9f (0xe0a0 bytes) 2024-09-19T21:35:17.866 INFO:teuthology.orchestra.run.smithi122.stdout:2466 trunc from 0xe3ef400 to 0xd6c5800 2024-09-19T21:35:17.922 INFO:teuthology.orchestra.run.smithi122.stdout:2467 trunc from 0xd6c5800 to 0x955c200 2024-09-19T21:35:17.922 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:17.917+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:39586 3.5 3:a4be4775:::rbd_data.1106846f0701.0000000000000144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:17.929075+0000", 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0074669259999999996, 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0074884210000000003, 2024-09-19T21:35:17.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for sub ops", 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 39586 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:17.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929075+0000", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929075+0000", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929074+0000", 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929082+0000", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8949999999999998e-06 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929100+0000", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7977000000000001e-05 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929113+0000", 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3519999999999999e-05 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929554+0000", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000440705 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929695+0000", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000141075 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 3", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.929866+0000", 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017093699999999999 2024-09-19T21:35:17.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:17.934158+0000", 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.004291441 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:35:17.943 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:17.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:35:17.948 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:17.957 INFO:teuthology.orchestra.run.smithi122.stdout:2468 read 0x7c4e400 thru 0x7c58aaa (0xa6ab bytes) 2024-09-19T21:35:17.960 INFO:teuthology.orchestra.run.smithi122.stdout:2469 read 0x2661a00 thru 0x266400c (0x260d bytes) 2024-09-19T21:35:17.961 INFO:teuthology.orchestra.run.smithi122.stdout:2471 write 0x49c8a00 thru 0x49cbc53 (0x3254 bytes) 2024-09-19T21:35:17.962 INFO:teuthology.orchestra.run.smithi122.stdout:2472 read 0x217b200 thru 0x2181b78 (0x6979 bytes) 2024-09-19T21:35:17.967 INFO:teuthology.orchestra.run.smithi122.stdout:2474 punch from 0x137cc00 to 0x137ce00, (0x200 bytes) 2024-09-19T21:35:17.968 INFO:teuthology.orchestra.run.smithi122.stdout:2476 write 0x3a50e00 thru 0x3a5ee46 (0xe047 bytes) 2024-09-19T21:35:17.968 INFO:teuthology.orchestra.run.smithi122.stdout:2477 read 0x3ac6800 thru 0x3ad3431 (0xcc32 bytes) 2024-09-19T21:35:17.968 INFO:teuthology.orchestra.run.smithi122.stdout:2478 read 0x82faa00 thru 0x83019e1 (0x6fe2 bytes) 2024-09-19T21:35:17.969 INFO:teuthology.orchestra.run.smithi122.stdout:2480 write 0x195d000 thru 0x195d0ee (0xef bytes) 2024-09-19T21:35:17.970 INFO:teuthology.orchestra.run.smithi122.stdout:2482 write 0x8541800 thru 0x854f626 (0xde27 bytes) 2024-09-19T21:35:17.982 INFO:teuthology.orchestra.run.smithi122.stdout:2486 trunc from 0x955c200 to 0x3f3b600 2024-09-19T21:35:18.023 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:18.017+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:18.055 INFO:teuthology.orchestra.run.smithi122.stdout:2489 read 0x1608a00 thru 0x1609d9f (0x13a0 bytes) 2024-09-19T21:35:18.058 INFO:teuthology.orchestra.run.smithi122.stdout:2496 read 0x151c000 thru 0x151c495 (0x496 bytes) 2024-09-19T21:35:18.059 INFO:teuthology.orchestra.run.smithi122.stdout:2497 punch from 0x3907000 to 0x3914200, (0xd200 bytes) 2024-09-19T21:35:18.060 INFO:teuthology.orchestra.run.smithi122.stdout:2498 read 0x2cee600 thru 0x2cef32d (0xd2e bytes) 2024-09-19T21:35:18.062 INFO:teuthology.orchestra.run.smithi122.stdout:2499 read 0x1077600 thru 0x1080dd6 (0x97d7 bytes) 2024-09-19T21:35:18.064 INFO:teuthology.orchestra.run.smithi122.stdout:2500 clone 16 order 22 su 262144 sc 10 2024-09-19T21:35:18.066 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:18.066 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:18.066 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:18.067 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:18.067 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:18.078 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:18.124 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:18.122+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:18.177 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 36.945669248999998, 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471641440999999, 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471611614, 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471601927999998, 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471593601999999, 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471583209999999, 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.471572714000001, 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.469128626, 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.443911239999998, 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:18.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:18.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15066 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6164480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5483165489999999, 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029164998000000001, 2024-09-19T21:35:18.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15066 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624793+0000", 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624797+0000", 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4580000000000002e-06 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624803+0000", 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6800000000000004e-06 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624809+0000", 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8969999999999996e-06 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653805+0000", 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028996141999999999 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653889+0000", 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3906000000000006e-05 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653959+0000", 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9604000000000006e-05 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15067 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6178816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5482879399999998, 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029307910999999999, 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15067 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624831+0000", 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2019999999999993e-06 2024-09-19T21:35:18.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624840+0000", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5639999999999997e-06 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624846+0000", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4740000000000002e-06 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653998+0000", 2024-09-19T21:35:18.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029151795000000001 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654062+0000", 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4289999999999996e-05 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654130+0000", 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8034000000000006e-05 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15068 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6193152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5482599390000003, 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029394144000000001, 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15068 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.232 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624853+0000", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8339999999999999e-06 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624860+0000", 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7689999999999999e-06 2024-09-19T21:35:18.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624866+0000", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0639999999999999e-06 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654143+0000", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277609999999999 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654189+0000", 2024-09-19T21:35:18.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5892e-05 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654245+0000", 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5383000000000002e-05 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15069 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5482395929999999, 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029542570000000001, 2024-09-19T21:35:18.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15069 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624870+0000", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624873+0000", 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624880+0000", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624886+0000", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654272+0000", 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029385674000000001 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654328+0000", 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6156999999999998e-05 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654413+0000", 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999998e-05 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15070 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5482194050000002, 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02965835, 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15070 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624894+0000", 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9900000000000002e-06 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624903+0000", 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1220000000000001e-06 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624917+0000", 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3973e-05 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654426+0000", 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029508869 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654475+0000", 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9274999999999998e-05 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654549+0000", 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4492000000000004e-05 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15071 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6236160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5481873720000001, 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029922358, 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15071 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624922+0000", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624926+0000", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2650000000000001e-06 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624933+0000", 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:35:18.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624939+0000", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9819999999999996e-06 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654596+0000", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029657246000000002 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654708+0000", 2024-09-19T21:35:18.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112194 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654845+0000", 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013685999999999999 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15072 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6250496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:18.249 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.54816596, 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030057148999999998, 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15072 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:18.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624947+0000", 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.326e-06 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624954+0000", 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6590000000000001e-06 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624959+0000", 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3630000000000002e-06 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654863+0000", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029903855999999999 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654917+0000", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4487000000000002e-05 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655002+0000", 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4300000000000003e-05 2024-09-19T21:35:18.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15073 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6264832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5481449569999999, 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030212118999999999, 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15073 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624964+0000", 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624968+0000", 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.16e-06 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624975+0000", 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9600000000000003e-06 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624981+0000", 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7980000000000002e-06 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655037+0000", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030056058 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655093+0000", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6223e-05 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655177+0000", 2024-09-19T21:35:18.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4654000000000006e-05 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15074 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5481247319999998, 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030446608, 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15074 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:18.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624985+0000", 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624988+0000", 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1370000000000002e-06 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625010+0000", 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1778000000000001e-05 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625017+0000", 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5259999999999999e-06 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655204+0000", 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030186983000000001 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655303+0000", 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655432+0000", 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012875899999999999 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18639 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 731136~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1511472610000002, 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029303412000000001, 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18639 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:18.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021964+0000", 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021967+0000", 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021970+0000", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2349999999999999e-06 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051070+0000", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029100002 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051159+0000", 2024-09-19T21:35:18.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9033e-05 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051266+0000", 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107069 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18640 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 745472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:18.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1511268910000001, 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029524404000000001, 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18640 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:18.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021985+0000", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9750000000000001e-06 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021988+0000", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9720000000000001e-06 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021991+0000", 2024-09-19T21:35:18.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6369999999999999e-06 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051317+0000", 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029326609 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051398+0000", 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0705999999999996e-05 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:18.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051508+0000", 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109948 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:18.273 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:18.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:18.275 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:18.269+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:18.375 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:18.374+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:18.390 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:18.390 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:18.390 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:18.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:18.476 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:18.469+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:18.506 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:18.506 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:18.506 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:18.506 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:18.506 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:18.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:18.577 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:18.574+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3675607899999997, 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:18.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:18.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:18.630 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3675472510000004, 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3675215109999996, 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.639 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3675072970000004, 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3674863799999999, 2024-09-19T21:35:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:18.649 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3674710040000004, 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3674560759999999, 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.654 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:18.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:18.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3674034559999999, 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:18.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:18.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3673538780000003, 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:18.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3673366629999997, 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.668 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:18.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.672 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3673175249999998, 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3673026769999996, 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.677 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:18.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3672796839999997, 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.686 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3665473920000002, 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:18.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3664970820000004, 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:18.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:18.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3664347699999997, 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:18.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:18.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3651041370000003, 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:18.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3649685590000002, 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:18.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.3648917999999997, 2024-09-19T21:35:18.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:18.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.364833076, 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:18.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:18.725 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:18.726 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:18.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:18.728 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:18.721+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:18.828 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:18.821+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:18.838 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:18.838 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:18.838 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:18.839 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:18.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:35:18.928 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:18.925+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:19.005 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:19.005 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:19.005 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:19.005 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:19.005 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:19.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:19.060 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:19.057+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:19.069 INFO:teuthology.orchestra.run.smithi122.stdout:leaving image image_client.0-clone15 intact 2024-09-19T21:35:19.122 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:19.122 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:19.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:19.122 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:19.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.717977052999998, 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:35:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:35:19.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.713823163999997, 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:35:19.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:35:19.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.713785760999997, 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.713767932000003, 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:35:19.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9026 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13496320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8539217170000004, 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061312130999999999, 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9026 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264255+0000", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5220000000000002e-06 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264260+0000", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2320000000000001e-06 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264265+0000", 2024-09-19T21:35:19.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7280000000000003e-06 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325377+0000", 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061111867 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325467+0000", 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0564999999999998e-05 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325565+0000", 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7547999999999995e-05 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9027 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13510656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8538715889999997, 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061437335000000003, 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.146 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9027 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264302+0000", 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264306+0000", 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7850000000000002e-06 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264313+0000", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5320000000000001e-06 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264319+0000", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6879999999999996e-06 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325612+0000", 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061292919000000001 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325674+0000", 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2638999999999994e-05 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325740+0000", 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5505000000000004e-05 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9390 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13225984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8417802759999997, 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061207497999999999, 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9390 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276348+0000", 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999542 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276400+0000", 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1529000000000001e-05 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276420+0000", 2024-09-19T21:35:19.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0203e-05 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276428+0000", 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9990000000000008e-06 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337400+0000", 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060972560000000002 2024-09-19T21:35:19.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337501+0000", 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010045800000000001 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337601+0000", 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100703 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9606 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12640256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8336319569999997, 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061298120999999997, 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9606 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284541+0000", 2024-09-19T21:35:19.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284547+0000", 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6470000000000003e-06 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284566+0000", 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8986000000000001e-05 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284578+0000", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1578e-05 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345651+0000", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061073167999999997 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345717+0000", 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6042000000000002e-05 2024-09-19T21:35:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345840+0000", 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012361699999999999 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9607 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12654592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8333115160000002, 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061227252000000003, 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9607 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284585+0000", 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997225 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284868+0000", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028231399999999998 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284890+0000", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2062e-05 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284896+0000", 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.246e-06 2024-09-19T21:35:19.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345877+0000", 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060981568999999999 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345976+0000", 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8826999999999994e-05 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346090+0000", 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113701 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9608 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12668928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8331826590000002, 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061372768000000001, 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9608 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284905+0000", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999132 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284997+0000", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2140999999999998e-05 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285012+0000", 2024-09-19T21:35:19.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4800999999999999e-05 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285032+0000", 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0514e-05 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346127+0000", 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061094674000000002 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346223+0000", 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5798000000000007e-05 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346364+0000", 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000141691 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9609 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12683264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.169 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8329220340000001, 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061363407000000002, 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9609 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285039+0000", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997869 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285258+0000", 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000219096 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285269+0000", 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285276+0000", 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5950000000000004e-06 2024-09-19T21:35:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346393+0000", 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061116960999999997 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346495+0000", 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010218299999999999 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346616+0000", 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120654 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9610 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12697600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8326926510000003, 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061447314000000003, 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9610 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285280+0000", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997988 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285487+0000", 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020666400000000001 2024-09-19T21:35:19.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285498+0000", 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1069e-05 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285506+0000", 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7449999999999995e-06 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346663+0000", 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157126999999999 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346785+0000", 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012234600000000001 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346929+0000", 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014350699999999999 2024-09-19T21:35:19.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9611 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12711936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8324283049999996, 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061431025, 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9611 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285512+0000", 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997663 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285751+0000", 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000238555 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285758+0000", 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6660000000000008e-06 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285764+0000", 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4709999999999998e-06 2024-09-19T21:35:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346967+0000", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061203562000000003 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347064+0000", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6366000000000005e-05 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347177+0000", 2024-09-19T21:35:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113276 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9612 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12726272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8321949049999997, 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061441227000000001, 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.184 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9612 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:19.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285768+0000", 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997888 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285984+0000", 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021622699999999999 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285995+0000", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0631e-05 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286011+0000", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5828000000000002e-05 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347226+0000", 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061214888000000002 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347346+0000", 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120116 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347421+0000", 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-05 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9613 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12740608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8319448520000003, 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061446091000000001, 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9613 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286023+0000", 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999794 2024-09-19T21:35:19.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286234+0000", 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000210609 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286241+0000", 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4710000000000001e-06 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286247+0000", 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347460+0000", 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061213103999999997 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347557+0000", 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7386999999999994e-05 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347675+0000", 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011809199999999999 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9614 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12754944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.83167934, 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061377640999999997, 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9614 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286257+0000", 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997625 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286501+0000", 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024361599999999999 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286511+0000", 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0614e-05 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286526+0000", 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4594e-05 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347724+0000", 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061198065000000003 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347794+0000", 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0146000000000002e-05 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347873+0000", 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8280000000000003e-05 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9615 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12769280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8314387070000002, 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061393000000000003, 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9615 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286534+0000", 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997988 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286742+0000", 2024-09-19T21:35:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020750099999999999 2024-09-19T21:35:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286754+0000", 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2123999999999999e-05 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286762+0000", 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347907+0000", 2024-09-19T21:35:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061144153999999999 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348009+0000", 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010231 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348129+0000", 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119628 2024-09-19T21:35:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9616 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12783616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.205 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8311612799999999, 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061308548999999997, 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9616 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286767+0000", 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997544 2024-09-19T21:35:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287019+0000", 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025142099999999999 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287041+0000", 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2878e-05 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287050+0000", 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.884e-06 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348161+0000", 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061110730000000002 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348226+0000", 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4586999999999997e-05 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348321+0000", 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5791000000000005e-05 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9617 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12797952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8309721679999997, 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061353525999999999, 2024-09-19T21:35:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9617 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287060+0000", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998579 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287208+0000", 2024-09-19T21:35:19.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014783 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287226+0000", 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8695e-05 2024-09-19T21:35:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287242+0000", 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5574e-05 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348348+0000", 2024-09-19T21:35:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061105680000000002 2024-09-19T21:35:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348447+0000", 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9481999999999994e-05 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348556+0000", 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108545 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.216 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9618 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12812288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.8306569230000003, 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061298677000000003, 2024-09-19T21:35:19.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9618 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287250+0000", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997325 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287524+0000", 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027381900000000002 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287538+0000", 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4313e-05 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287547+0000", 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9649999999999997e-06 2024-09-19T21:35:19.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348597+0000", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061049919000000001 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348676+0000", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8997000000000003e-05 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348816+0000", 2024-09-19T21:35:19.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014009999999999999 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.222 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:19.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:19.224 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:19.217+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:19.324 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:19.321+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:19.343 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:19.343 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:19.343 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:19.343 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:19.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:19.425 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:19.421+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:19.465 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:19.465 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:19.465 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:19.465 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:19.465 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:19.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:19.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:19.526 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:19.522+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.839714955000002, 2024-09-19T21:35:19.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:19.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:19.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38478 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3237888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:19.602 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.0030135320000002, 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050662393999999999, 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38478 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585326+0000", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.36e-06 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585339+0000", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2836999999999999e-05 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585349+0000", 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0227999999999999e-05 2024-09-19T21:35:19.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635814+0000", 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050465015000000002 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635883+0000", 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8490000000000003e-05 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635986+0000", 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102972 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38479 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 2.0028044519999999, 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050617212000000002, 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38479 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585354+0000", 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998221 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585538+0000", 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000184007 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585548+0000", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585555+0000", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1380000000000001e-06 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636018+0000", 2024-09-19T21:35:19.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050463348999999998 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636091+0000", 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3081e-05 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636150+0000", 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8387000000000002e-05 2024-09-19T21:35:19.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38745 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3299328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9860506959999999, 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050507966000000001, 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38745 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:19.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602096+0000", 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998102 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602292+0000", 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019607700000000001 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602303+0000", 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0248000000000001e-05 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602320+0000", 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7898e-05 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652572+0000", 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050251704000000001 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652684+0000", 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011210299999999999 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652794+0000", 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109832 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38746 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3313664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9858755729999999, 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050552997000000002, 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38746 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602337+0000", 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999876 2024-09-19T21:35:19.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602467+0000", 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000129827 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602478+0000", 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.057e-05 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602503+0000", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5826e-05 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652823+0000", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050319567000000003 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652916+0000", 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2750999999999993e-05 2024-09-19T21:35:19.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653014+0000", 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8435000000000002e-05 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38747 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3328000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:19.621 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9857761759999999, 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050666628999999998, 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38747 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602511+0000", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999504 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602566+0000", 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4653e-05 2024-09-19T21:35:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602574+0000", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1799999999999996e-06 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602581+0000", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653042+0000", 2024-09-19T21:35:19.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050461113000000002 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653135+0000", 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2662000000000002e-05 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653227+0000", 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2315e-05 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.625 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38748 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.985588321, 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050685503999999999, 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38748 2024-09-19T21:35:19.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602587+0000", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998384 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602755+0000", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000167875 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602781+0000", 2024-09-19T21:35:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6815e-05 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602800+0000", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8644999999999999e-05 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653253+0000", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050452664000000001 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653338+0000", 2024-09-19T21:35:19.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5718999999999997e-05 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653434+0000", 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5524000000000002e-05 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38749 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3356672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9853158479999999, 2024-09-19T21:35:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050628085000000003, 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38749 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602862+0000", 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998407 2024-09-19T21:35:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603026+0000", 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016469600000000001 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603037+0000", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0208e-05 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603049+0000", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2133999999999999e-05 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653466+0000", 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050417464000000002 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653553+0000", 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6384999999999994e-05 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653649+0000", 2024-09-19T21:35:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6376e-05 2024-09-19T21:35:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38750 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3371008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9850289239999999, 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050551984000000001, 2024-09-19T21:35:19.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38750 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603056+0000", 2024-09-19T21:35:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997478 2024-09-19T21:35:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603313+0000", 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025696099999999999 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603333+0000", 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0117000000000001e-05 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603341+0000", 2024-09-19T21:35:19.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653674+0000", 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050333152999999999 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653765+0000", 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1237999999999996e-05 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653860+0000", 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4347999999999999e-05 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38751 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3385344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.984807319, 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050533023000000003, 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38751 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603350+0000", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998207 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603534+0000", 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018449199999999999 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603544+0000", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7489999999999993e-06 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603552+0000", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8269999999999998e-06 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653888+0000", 2024-09-19T21:35:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050335574000000001 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653975+0000", 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7108999999999996e-05 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654062+0000", 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7756000000000005e-05 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38754 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3428352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9842265349999999, 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050586417000000002, 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38754 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604109+0000", 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604114+0000", 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604121+0000", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7170000000000008e-06 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604127+0000", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9889999999999997e-06 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654520+0000", 2024-09-19T21:35:19.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050392827000000001 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654605+0000", 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5191999999999993e-05 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654697+0000", 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1293999999999997e-05 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38755 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3442688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9840721509999999, 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050639968, 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38755 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604132+0000", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998674 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604276+0000", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014431399999999999 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604295+0000", 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8457999999999998e-05 2024-09-19T21:35:19.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604303+0000", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9820000000000008e-06 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654725+0000", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050422012000000002 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654811+0000", 2024-09-19T21:35:19.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5921000000000006e-05 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654905+0000", 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3906000000000005e-05 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38775 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3729408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9799056749999999, 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050773897999999998, 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38775 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608432+0000", 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.424e-06 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608435+0000", 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608438+0000", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.774e-06 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.658966+0000", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050528196999999997 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659086+0000", 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119994 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659205+0000", 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118849 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38789 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3930112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9761606029999998, 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050834478000000002, 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38789 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612175+0000", 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612178+0000", 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9519999999999999e-06 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612185+0000", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2740000000000004e-06 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612192+0000", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662685+0000", 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050493581000000003 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662845+0000", 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016009299999999999 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663011+0000", 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016543800000000001 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38790 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3944448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.976129069, 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051021812, 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38790 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612207+0000", 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612210+0000", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000001e-06 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612217+0000", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3159999999999999e-06 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612223+0000", 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:19.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663043+0000", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050820585000000001 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663124+0000", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0409999999999998e-05 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663230+0000", 2024-09-19T21:35:19.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105659 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38791 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3958784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.976109238, 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051185702, 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38791 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612227+0000", 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612238+0000", 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1326e-05 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612244+0000", 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1430000000000003e-06 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612250+0000", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5249999999999996e-06 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663257+0000", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051007515000000003 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663335+0000", 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7265e-05 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663413+0000", 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.852e-05 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38804 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4145152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9723669419999998, 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050530841, 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38804 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615969+0000", 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615973+0000", 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5209999999999998e-06 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615981+0000", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5269999999999998e-06 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615987+0000", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666333+0000", 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050346683000000003 2024-09-19T21:35:19.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666413+0000", 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9437000000000005e-05 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666501+0000", 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7766e-05 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.679 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38805 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.972345432, 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050694516000000002, 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38805 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615994+0000", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616006+0000", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2201999999999999e-05 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616015+0000", 2024-09-19T21:35:19.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4730000000000006e-06 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666526+0000", 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050510477999999998 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666598+0000", 2024-09-19T21:35:19.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1769999999999999e-05 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666686+0000", 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8330000000000003e-05 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38806 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4173824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:19.684 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.9723164419999999, 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0508669, 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38806 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:19.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616023+0000", 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2260000000000001e-06 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616031+0000", 2024-09-19T21:35:19.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8760000000000005e-06 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616036+0000", 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.848e-06 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666710+0000", 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050674286999999998 2024-09-19T21:35:19.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666789+0000", 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8916000000000001e-05 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666887+0000", 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8210999999999999e-05 2024-09-19T21:35:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38807 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4188160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 1.972296627, 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051052267999999998, 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38807 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616042+0000", 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620000000000001e-06 2024-09-19T21:35:19.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616048+0000", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0310000000000004e-06 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616059+0000", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0855e-05 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666927+0000", 2024-09-19T21:35:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050868501000000003 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667017+0000", 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9625999999999999e-05 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667092+0000", 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5337000000000001e-05 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:19.697 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:19.697 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:19.698 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:19.698 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:19.698 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:19.694+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:19.699 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:19.716 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:19.731 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-19T21:35:19.741 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:19.765 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:19.788 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:19.790 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:19.799 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:19.793+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:19.812 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:19.812 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:19.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:19.899 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:19.893+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:19.931 INFO:teuthology.orchestra.run.smithi122.stdout:2501 write 0xdda7800 thru 0xdda86f8 (0xef9 bytes) 2024-09-19T21:35:19.935 INFO:teuthology.orchestra.run.smithi122.stdout:2502 read 0xb84e200 thru 0xb85a44b (0xc24c bytes) 2024-09-19T21:35:19.935 INFO:teuthology.orchestra.run.smithi122.stdout:2505 punch from 0x2b08000 to 0x2b15400, (0xd400 bytes) 2024-09-19T21:35:19.940 INFO:teuthology.orchestra.run.smithi122.stdout:2508 read 0x1ba3c00 thru 0x1bad50b (0x990c bytes) 2024-09-19T21:35:19.960 INFO:teuthology.orchestra.run.smithi122.stdout:2509 trunc from 0xdda8800 to 0x66d6200 2024-09-19T21:35:19.982 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:19.982 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:19.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:20.000 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:19.997+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:20.014 INFO:teuthology.orchestra.run.smithi122.stdout:2510 write 0x34a3e00 thru 0x34a58a7 (0x1aa8 bytes) 2024-09-19T21:35:20.087 INFO:teuthology.orchestra.run.smithi122.stdout:2512 write 0xd7cc600 thru 0xd7d40c5 (0x7ac6 bytes) 2024-09-19T21:35:20.089 INFO:teuthology.orchestra.run.smithi122.stdout:2513 write 0x8edf400 thru 0x8eea88f (0xb490 bytes) 2024-09-19T21:35:20.089 INFO:teuthology.orchestra.run.smithi122.stdout:2515 read 0x323aa00 thru 0x324005b (0x565c bytes) 2024-09-19T21:35:20.091 INFO:teuthology.orchestra.run.smithi122.stdout:2516 write 0xc117600 thru 0xc11ab3c (0x353d bytes) 2024-09-19T21:35:20.091 INFO:teuthology.orchestra.run.smithi122.stdout:2521 read 0x2aa8000 thru 0x2aa82f2 (0x2f3 bytes) 2024-09-19T21:35:20.092 INFO:teuthology.orchestra.run.smithi122.stdout:2523 read 0xc4b7600 thru 0xc4c4f80 (0xd981 bytes) 2024-09-19T21:35:20.092 INFO:teuthology.orchestra.run.smithi122.stdout:2526 read 0x747c200 thru 0x748c1e7 (0xffe8 bytes) 2024-09-19T21:35:20.093 INFO:teuthology.orchestra.run.smithi122.stdout:2527 write 0xb5e3c00 thru 0xb5e484a (0xc4b bytes) 2024-09-19T21:35:20.093 INFO:teuthology.orchestra.run.smithi122.stdout:2531 punch from 0x7e9bc00 to 0x7ea9e00, (0xe200 bytes) 2024-09-19T21:35:20.095 INFO:teuthology.orchestra.run.smithi122.stdout:2532 write 0x8c20c00 thru 0x8c2fd9f (0xf1a0 bytes) 2024-09-19T21:35:20.095 INFO:teuthology.orchestra.run.smithi122.stdout:2533 read 0xe57c00 thru 0xe64254 (0xc655 bytes) 2024-09-19T21:35:20.101 INFO:teuthology.orchestra.run.smithi122.stdout:2537 read 0xb81c800 thru 0xb81ff77 (0x3778 bytes) 2024-09-19T21:35:20.101 INFO:teuthology.orchestra.run.smithi122.stdout:2538 write 0x7683e00 thru 0x7690404 (0xc605 bytes) 2024-09-19T21:35:20.101 INFO:teuthology.orchestra.run.smithi122.stdout:2539 write 0xcdebe00 thru 0xcdf8a41 (0xcc42 bytes) 2024-09-19T21:35:20.102 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:20.097+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:20.104 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.104 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:20.141 INFO:teuthology.orchestra.run.smithi122.stdout:2540 trunc from 0xd7d4200 to 0x193de00 2024-09-19T21:35:20.202 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:20.197+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:20.248 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.249 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:20.302 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:20.297+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:20.393 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.393 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.404 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:20.397+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:20.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:20.504 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:20.502+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:20.535 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.535 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:20.605 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:20.602+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:20.705 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:20.702+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:20.717 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.717 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:20.738 INFO:teuthology.orchestra.run.smithi122.stdout:2541 write 0x18f9400 thru 0x18feea5 (0x5aa6 bytes) 2024-09-19T21:35:20.744 INFO:teuthology.orchestra.run.smithi122.stdout:2542 read 0x94a00 thru 0xa3b4e (0xf14f bytes) 2024-09-19T21:35:20.776 INFO:teuthology.orchestra.run.smithi122.stdout:2544 write 0x4b73400 thru 0x4b77560 (0x4161 bytes) 2024-09-19T21:35:20.801 INFO:teuthology.orchestra.run.smithi122.stdout:2545 write 0x55b6200 thru 0x55b948e (0x328f bytes) 2024-09-19T21:35:20.807 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:20.802+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:20.845 INFO:teuthology.orchestra.run.smithi122.stdout:2548 write 0xd24fe00 thru 0xd258f08 (0x9109 bytes) 2024-09-19T21:35:20.848 INFO:teuthology.orchestra.run.smithi122.stdout:2549 write 0xcb42200 thru 0xcb44a87 (0x2888 bytes) 2024-09-19T21:35:20.848 INFO:teuthology.orchestra.run.smithi122.stdout:2550 punch from 0x1244200 to 0x1244800, (0x600 bytes) 2024-09-19T21:35:20.849 INFO:teuthology.orchestra.run.smithi122.stdout:2551 write 0x7d0ae00 thru 0x7d17fe0 (0xd1e1 bytes) 2024-09-19T21:35:20.849 INFO:teuthology.orchestra.run.smithi122.stdout:2555 write 0x54ca000 thru 0x54ce2ed (0x42ee bytes) 2024-09-19T21:35:20.849 INFO:teuthology.orchestra.run.smithi122.stdout:2560 read 0x3198e00 thru 0x3199008 (0x209 bytes) 2024-09-19T21:35:20.864 INFO:teuthology.orchestra.run.smithi122.stdout:2561 trunc from 0xd259000 to 0x6541200 2024-09-19T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:20.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:20.908 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:20.901+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:20.910 INFO:teuthology.orchestra.run.smithi122.stdout:2563 trunc from 0x6541200 to 0x24d0a00 2024-09-19T21:35:20.923 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:35:20.943 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:20.943 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:20.943 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-19T21:35:20.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:35:20.945 INFO:teuthology.orchestra.run.smithi122.stdout:2565 read 0xce1800 thru 0xce4f4e (0x374f bytes) 2024-09-19T21:35:20.948 INFO:teuthology.orchestra.run.smithi122.stdout:2566 punch from 0x1d63600 to 0x1d72e00, (0xf800 bytes) 2024-09-19T21:35:20.972 INFO:teuthology.orchestra.run.smithi122.stdout:2567 trunc from 0x24d0a00 to 0xd491800 2024-09-19T21:35:20.993 INFO:teuthology.orchestra.run.smithi122.stdout:2568 write 0x4fc3e00 thru 0x4fc5232 (0x1433 bytes) 2024-09-19T21:35:20.994 INFO:teuthology.orchestra.run.smithi122.stdout:2570 read 0x455c600 thru 0x4566ebe (0xa8bf bytes) 2024-09-19T21:35:21.003 INFO:teuthology.orchestra.run.smithi122.stdout:2571 trunc from 0xd491800 to 0xe79fe00 2024-09-19T21:35:21.003 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.004 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.009 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:21.006+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:21.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:21.031 INFO:teuthology.orchestra.run.smithi122.stdout:2575 punch from 0x6192400 to 0x61a1600, (0xf200 bytes) 2024-09-19T21:35:21.032 INFO:teuthology.orchestra.run.smithi122.stdout:2576 read 0x7cdf200 thru 0x7cead0a (0xbb0b bytes) 2024-09-19T21:35:21.033 INFO:teuthology.orchestra.run.smithi122.stdout:2577 write 0x1d49600 thru 0x1d4b531 (0x1f32 bytes) 2024-09-19T21:35:21.034 INFO:teuthology.orchestra.run.smithi122.stdout:2578 read 0x2aab200 thru 0x2ab398e (0x878f bytes) 2024-09-19T21:35:21.034 INFO:teuthology.orchestra.run.smithi122.stdout:2579 punch from 0x3db5e00 to 0x3dc2800, (0xca00 bytes) 2024-09-19T21:35:21.035 INFO:teuthology.orchestra.run.smithi122.stdout:2580 read 0xd5b9000 thru 0xd5c4c6f (0xbc70 bytes) 2024-09-19T21:35:21.035 INFO:teuthology.orchestra.run.smithi122.stdout:2582 read 0x9d6b000 thru 0x9d75f28 (0xaf29 bytes) 2024-09-19T21:35:21.036 INFO:teuthology.orchestra.run.smithi122.stdout:2584 read 0xe6be600 thru 0xe6c9a7b (0xb47c bytes) 2024-09-19T21:35:21.036 INFO:teuthology.orchestra.run.smithi122.stdout:2585 punch from 0x1c16a00 to 0x1c24c00, (0xe200 bytes) 2024-09-19T21:35:21.036 INFO:teuthology.orchestra.run.smithi122.stdout:2588 read 0x9e6fa00 thru 0x9e6fba3 (0x1a4 bytes) 2024-09-19T21:35:21.036 INFO:teuthology.orchestra.run.smithi122.stdout:2589 write 0xd9df000 thru 0xd9eb750 (0xc751 bytes) 2024-09-19T21:35:21.050 INFO:teuthology.orchestra.run.smithi122.stdout:2591 trunc from 0xe79fe00 to 0x947dc00 2024-09-19T21:35:21.077 INFO:teuthology.orchestra.run.smithi122.stdout:2592 read 0x11f9e00 thru 0x12025fa (0x87fb bytes) 2024-09-19T21:35:21.080 INFO:teuthology.orchestra.run.smithi122.stdout:2595 read 0x1e4b400 thru 0x1e57602 (0xc203 bytes) 2024-09-19T21:35:21.082 INFO:teuthology.orchestra.run.smithi122.stdout:2596 write 0x56fba00 thru 0x5708af8 (0xd0f9 bytes) 2024-09-19T21:35:21.082 INFO:teuthology.orchestra.run.smithi122.stdout:2597 read 0xf09400 thru 0xf0f003 (0x5c04 bytes) 2024-09-19T21:35:21.092 INFO:teuthology.orchestra.run.smithi122.stdout:2600 trunc from 0x947dc00 to 0x6233600 2024-09-19T21:35:21.110 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:21.105+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:21.143 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.143 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:21.161 INFO:teuthology.orchestra.run.smithi122.stdout:2601 write 0xd1f8200 thru 0xd200c3e (0x8a3f bytes) 2024-09-19T21:35:21.180 INFO:teuthology.orchestra.run.smithi122.stdout:2603 trunc from 0xd200e00 to 0x4fd1600 2024-09-19T21:35:21.211 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:21.205+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:21.221 INFO:teuthology.orchestra.run.smithi122.stdout:2604 punch from 0x1e99c00 to 0x1e9d600, (0x3a00 bytes) 2024-09-19T21:35:21.224 INFO:teuthology.orchestra.run.smithi122.stdout:2605 trunc from 0x4fd1600 to 0x4817800 2024-09-19T21:35:21.247 INFO:teuthology.orchestra.run.smithi122.stdout:2606 read 0x140b800 thru 0x140e4f0 (0x2cf1 bytes) 2024-09-19T21:35:21.249 INFO:teuthology.orchestra.run.smithi122.stdout:2609 read 0x3b39800 thru 0x3b3aef3 (0x16f4 bytes) 2024-09-19T21:35:21.250 INFO:teuthology.orchestra.run.smithi122.stdout:2612 write 0x448e800 thru 0x449d3e6 (0xebe7 bytes) 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 54, 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:35:18.889867+0000", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:34:04.640393+0000", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:35:21.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5801984, 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "noscrub", 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:35:21.261 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "jewel", 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:35:21.262 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:21.264 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:35:21.265 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:21.266 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:21.267 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:21.267 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:21.267 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:21.268 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:35:21.269 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:21.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:35:21.271 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:21.272 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:21.273 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.274 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.275 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.25, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.25, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:35:21.276 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:21.277 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.278 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "54", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 19, 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 54, 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:21.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:21.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:21.281 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.1299999952316284, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.1299999952316284, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:21.283 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.285 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.288 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:21.289 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:35:21.290 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.291 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.294 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.295 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:35:21.297 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:35:21.298 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.299 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 24, 2024-09-19T21:35:21.301 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.304 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.305 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.307 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:35:21.311 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:35:21.312 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:35:21.313 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.316 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:35:21.318 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.319 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.320 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.322 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:21.323 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:35:21.324 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:35:21.325 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:21.326 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:21.329 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.329 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.330 INFO:tasks.thrashosds.thrasher:j is {'epoch': 54, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:35:18.889867+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:34:04.640393+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 54, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:35:21.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:35:21.330 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:21.326+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:21.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2616 write 0xd3b8000 thru 0xd3c581f (0xd820 bytes) 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2617 write 0xa1c00 thru 0xa3924 (0x1d25 bytes) 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2619 write 0xd039400 thru 0xd03cf81 (0x3b82 bytes) 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2620 punch from 0x246ac00 to 0x2477800, (0xcc00 bytes) 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2623 punch from 0x407e600 to 0x4087000, (0x8a00 bytes) 2024-09-19T21:35:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:2625 punch from 0x4f3c00 to 0x4f9c00, (0x6000 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2627 read 0xc40ae00 thru 0xc40b5b8 (0x7b9 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2628 read 0x5669c00 thru 0x566eca8 (0x50a9 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2630 write 0x4975400 thru 0x49840cd (0xecce bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2631 write 0xc4d6a00 thru 0xc4d940c (0x2a0d bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2633 punch from 0x8924c00 to 0x8928400, (0x3800 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2635 write 0x3822c00 thru 0x3829a5c (0x6e5d bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2636 punch from 0x90f4400 to 0x90fda00, (0x9600 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2638 read 0x3234c00 thru 0x323a1f1 (0x55f2 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2639 read 0x5fde200 thru 0x5fe2747 (0x4548 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2640 read 0x6dbf800 thru 0x6dc013f (0x940 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2642 read 0xc93e200 thru 0xc94d581 (0xf382 bytes) 2024-09-19T21:35:21.333 INFO:teuthology.orchestra.run.smithi122.stdout:2643 trunc from 0xd3c5a00 to 0x55b8a00 2024-09-19T21:35:21.353 INFO:teuthology.orchestra.run.smithi122.stdout:2644 read 0xfebc00 thru 0xff89ad (0xcdae bytes) 2024-09-19T21:35:21.369 INFO:teuthology.orchestra.run.smithi122.stdout:2645 trunc from 0x55b8a00 to 0xb1da600 2024-09-19T21:35:21.388 INFO:teuthology.orchestra.run.smithi122.stdout:2646 read 0x48b5e00 thru 0x48be643 (0x8844 bytes) 2024-09-19T21:35:21.390 INFO:teuthology.orchestra.run.smithi122.stdout:2648 write 0x8221600 thru 0x822296a (0x136b bytes) 2024-09-19T21:35:21.396 INFO:teuthology.orchestra.run.smithi122.stdout:2650 trunc from 0xb1da600 to 0x8982400 2024-09-19T21:35:21.422 INFO:teuthology.orchestra.run.smithi122.stdout:2651 read 0x6960c00 thru 0x696ec73 (0xe074 bytes) 2024-09-19T21:35:21.425 INFO:teuthology.orchestra.run.smithi122.stdout:2654 write 0x3568e00 thru 0x3577d38 (0xef39 bytes) 2024-09-19T21:35:21.426 INFO:teuthology.orchestra.run.smithi122.stdout:2655 write 0x19d200 thru 0x1ab074 (0xde75 bytes) 2024-09-19T21:35:21.427 INFO:teuthology.orchestra.run.smithi122.stdout:2656 read 0x7a3800 thru 0x7af922 (0xc123 bytes) 2024-09-19T21:35:21.428 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:21.426+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:21.430 INFO:teuthology.orchestra.run.smithi122.stdout:2657 write 0x7094400 thru 0x7096311 (0x1f12 bytes) 2024-09-19T21:35:21.430 INFO:teuthology.orchestra.run.smithi122.stdout:2658 clone 17 order 18 su 32768 sc 13 2024-09-19T21:35:21.453 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.453 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:21.529 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:21.526+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:21.567 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.568 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:21.630 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:21.625+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:21.655 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:21.657 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:35:21.670 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-19T21:35:20.814869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":996773787,"num_objects":645,"num_object_clones":174,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":645,"num_whiteouts":202,"num_read":120897,"num_read_kb":425150,"num_write":6322,"num_write_kb":93855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5853,"ondisk_log_size":5853,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":334452,"kb_used_data":119044,"kb_used_omap":9,"kb_used_meta":215286,"kb_avail":565896588,"statfs":{"total":579820584960,"available":579478106112,"internally_reserved":0,"allocated":121901056,"data_stored":118879824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":220453542},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,47,0,54,90,53],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":119403167,"num_objects":109,"num_object_clones":10,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":9,"num_read":52023,"num_read_kb":87256,"num_write":2073,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002695"},"pg_stats":[{"pgid":"3.1f","version":"53'213","reported_seq":1833,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.859749+0000","last_change":"2024-09-19T21:34:32.012011+0000","last_active":"2024-09-19T21:35:17.859749+0000","last_peered":"2024-09-19T21:35:17.859749+0000","last_clean":"2024-09-19T21:35:17.859749+0000","last_became_active":"2024-09-19T21:34:30.013688+0000","last_became_peered":"2024-09-19T21:34:30.013688+0000","last_unstale":"2024-09-19T21:35:17.859749+0000","last_undegraded":"2024-09-19T21:35:17.859749+0000","last_fullsized":"2024-09-19T21:35:17.859749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:27:13.050028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":35160110,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":3201,"num_read_kb":21408,"num_write":205,"num_write_kb":17486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"53'72","reported_seq":519,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:16.563900+0000","last_change":"2024-09-19T21:34:32.006487+0000","last_active":"2024-09-19T21:35:16.563900+0000","last_peered":"2024-09-19T21:35:16.563900+0000","last_clean":"2024-09-19T21:35:16.563900+0000","last_became_active":"2024-09-19T21:34:30.011721+0000","last_became_peered":"2024-09-19T21:34:30.011721+0000","last_unstale":"2024-09-19T21:35:16.563900+0000","last_undegraded":"2024-09-19T21:35:16.563900+0000","last_fullsized":"2024-09-19T21:35:16.563900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:21.923196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":11507728,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":8,"num_read":442,"num_read_kb":1859,"num_write":55,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"53'545","reported_seq":2515,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.038344+0000","last_change":"2024-09-19T21:34:32.024269+0000","last_active":"2024-09-19T21:35:18.038344+0000","last_peered":"2024-09-19T21:35:18.038344+0000","last_clean":"2024-09-19T21:35:18.038344+0000","last_became_active":"2024-09-19T21:34:30.010774+0000","last_became_peered":"2024-09-19T21:34:30.010774+0000","last_unstale":"2024-09-19T21:35:18.038344+0000","last_undegraded":"2024-09-19T21:35:18.038344+0000","last_fullsized":"2024-09-19T21:35:18.038344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:59.647828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":57700008,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":4596,"num_read_kb":20309,"num_write":959,"num_write_kb":11020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"53'252","reported_seq":9660,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.931586+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:35:17.931586+0000","last_peered":"2024-09-19T21:35:17.931586+0000","last_clean":"2024-09-19T21:35:17.931586+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:35:17.931586+0000","last_undegraded":"2024-09-19T21:35:17.931586+0000","last_fullsized":"2024-09-19T21:35:17.931586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":9899556,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":10627,"num_read_kb":14640,"num_write":195,"num_write_kb":1197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"53'198","reported_seq":3091,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:16.226112+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:35:16.226112+0000","last_peered":"2024-09-19T21:35:16.226112+0000","last_clean":"2024-09-19T21:35:16.226112+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:35:16.226112+0000","last_undegraded":"2024-09-19T21:35:16.226112+0000","last_fullsized":"2024-09-19T21:35:16.226112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":116983443,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":3034,"num_read_kb":21199,"num_write":202,"num_write_kb":5027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"53'104","reported_seq":2482,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.060669+0000","last_change":"2024-09-19T21:34:32.001087+0000","last_active":"2024-09-19T21:35:18.060669+0000","last_peered":"2024-09-19T21:35:18.060669+0000","last_clean":"2024-09-19T21:35:18.060669+0000","last_became_active":"2024-09-19T21:34:30.002669+0000","last_became_peered":"2024-09-19T21:34:30.002669+0000","last_unstale":"2024-09-19T21:35:18.060669+0000","last_undegraded":"2024-09-19T21:35:18.060669+0000","last_fullsized":"2024-09-19T21:35:18.060669+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:54:28.569315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":12968008,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":2393,"num_read_kb":16010,"num_write":98,"num_write_kb":1142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"53'225","reported_seq":608,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.859959+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:35:17.859959+0000","last_peered":"2024-09-19T21:35:17.859959+0000","last_clean":"2024-09-19T21:35:17.859959+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:35:17.859959+0000","last_undegraded":"2024-09-19T21:35:17.859959+0000","last_fullsized":"2024-09-19T21:35:17.859959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":18688146,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1146,"num_read_kb":4257,"num_write":314,"num_write_kb":978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"53'120","reported_seq":4426,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:16.792371+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:35:16.792371+0000","last_peered":"2024-09-19T21:35:16.792371+0000","last_clean":"2024-09-19T21:35:16.792371+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:35:16.792371+0000","last_undegraded":"2024-09-19T21:35:16.792371+0000","last_fullsized":"2024-09-19T21:35:16.792371+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":23172654,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":5067,"num_read_kb":11990,"num_write":82,"num_write_kb":1083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"53'94","reported_seq":3334,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.024461+0000","last_change":"2024-09-19T21:34:32.012032+0000","last_active":"2024-09-19T21:35:18.024461+0000","last_peered":"2024-09-19T21:35:18.024461+0000","last_clean":"2024-09-19T21:35:18.024461+0000","last_became_active":"2024-09-19T21:34:30.011066+0000","last_became_peered":"2024-09-19T21:34:30.011066+0000","last_unstale":"2024-09-19T21:35:18.024461+0000","last_undegraded":"2024-09-19T21:35:18.024461+0000","last_fullsized":"2024-09-19T21:35:18.024461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:40:19.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":6676992,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":3218,"num_read_kb":5828,"num_write":83,"num_write_kb":549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"53'113","reported_seq":5846,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.077412+0000","last_change":"2024-09-19T21:34:32.024453+0000","last_active":"2024-09-19T21:35:18.077412+0000","last_peered":"2024-09-19T21:35:18.077412+0000","last_clean":"2024-09-19T21:35:18.077412+0000","last_became_active":"2024-09-19T21:34:30.006875+0000","last_became_peered":"2024-09-19T21:34:30.006875+0000","last_unstale":"2024-09-19T21:35:18.077412+0000","last_undegraded":"2024-09-19T21:35:18.077412+0000","last_fullsized":"2024-09-19T21:35:18.077412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:26:13.507158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":36966952,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":5756,"num_read_kb":17446,"num_write":114,"num_write_kb":2606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"53'66","reported_seq":3537,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.032865+0000","last_change":"2024-09-19T21:34:32.006201+0000","last_active":"2024-09-19T21:35:18.032865+0000","last_peered":"2024-09-19T21:35:18.032865+0000","last_clean":"2024-09-19T21:35:18.032865+0000","last_became_active":"2024-09-19T21:34:30.008159+0000","last_became_peered":"2024-09-19T21:34:30.008159+0000","last_unstale":"2024-09-19T21:35:18.032865+0000","last_undegraded":"2024-09-19T21:35:18.032865+0000","last_fullsized":"2024-09-19T21:35:18.032865+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:44:57.994848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":6677707,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":3463,"num_read_kb":3850,"num_write":54,"num_write_kb":416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"53'112","reported_seq":5284,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.025878+0000","last_change":"2024-09-19T21:34:32.002842+0000","last_active":"2024-09-19T21:35:18.025878+0000","last_peered":"2024-09-19T21:35:18.025878+0000","last_clean":"2024-09-19T21:35:18.025878+0000","last_became_active":"2024-09-19T21:34:30.010116+0000","last_became_peered":"2024-09-19T21:34:30.010116+0000","last_unstale":"2024-09-19T21:35:18.025878+0000","last_undegraded":"2024-09-19T21:35:18.025878+0000","last_fullsized":"2024-09-19T21:35:18.025878+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:38:12.596445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":33895440,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":5208,"num_read_kb":13808,"num_write":111,"num_write_kb":2714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.359656+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:35:12.359656+0000","last_peered":"2024-09-19T21:35:12.359656+0000","last_clean":"2024-09-19T21:35:12.359656+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:35:12.359656+0000","last_undegraded":"2024-09-19T21:35:12.359656+0000","last_fullsized":"2024-09-19T21:35:12.359656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"53'250","reported_seq":5424,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.061132+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:35:18.061132+0000","last_peered":"2024-09-19T21:35:18.061132+0000","last_clean":"2024-09-19T21:35:18.061132+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:35:18.061132+0000","last_undegraded":"2024-09-19T21:35:18.061132+0000","last_fullsized":"2024-09-19T21:35:18.061132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":47159885,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":5576,"num_read_kb":12939,"num_write":235,"num_write_kb":4100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.438162+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:35:12.438162+0000","last_peered":"2024-09-19T21:35:12.438162+0000","last_clean":"2024-09-19T21:35:12.438162+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:35:12.438162+0000","last_undegraded":"2024-09-19T21:35:12.438162+0000","last_fullsized":"2024-09-19T21:35:12.438162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"53'348","reported_seq":3341,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.073713+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:35:18.073713+0000","last_peered":"2024-09-19T21:35:18.073713+0000","last_clean":"2024-09-19T21:35:18.073713+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:35:18.073713+0000","last_undegraded":"2024-09-19T21:35:18.073713+0000","last_fullsized":"2024-09-19T21:35:18.073713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":18331678,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":4136,"num_read_kb":8118,"num_write":432,"num_write_kb":2324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.359386+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:35:12.359386+0000","last_peered":"2024-09-19T21:35:12.359386+0000","last_clean":"2024-09-19T21:35:12.359386+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:35:12.359386+0000","last_undegraded":"2024-09-19T21:35:12.359386+0000","last_fullsized":"2024-09-19T21:35:12.359386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"53'126","reported_seq":5714,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.028096+0000","last_change":"2024-09-19T21:34:32.024327+0000","last_active":"2024-09-19T21:35:18.028096+0000","last_peered":"2024-09-19T21:35:18.028096+0000","last_clean":"2024-09-19T21:35:18.028096+0000","last_became_active":"2024-09-19T21:34:30.007041+0000","last_became_peered":"2024-09-19T21:34:30.007041+0000","last_unstale":"2024-09-19T21:35:18.028096+0000","last_undegraded":"2024-09-19T21:35:18.028096+0000","last_fullsized":"2024-09-19T21:35:18.028096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:31:26.974439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":32402944,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":5585,"num_read_kb":15562,"num_write":104,"num_write_kb":2074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.362178+0000","last_change":"2024-09-19T21:34:31.005184+0000","last_active":"2024-09-19T21:35:12.362178+0000","last_peered":"2024-09-19T21:35:12.362178+0000","last_clean":"2024-09-19T21:35:12.362178+0000","last_became_active":"2024-09-19T21:34:31.004633+0000","last_became_peered":"2024-09-19T21:34:31.004633+0000","last_unstale":"2024-09-19T21:35:12.362178+0000","last_undegraded":"2024-09-19T21:35:12.362178+0000","last_fullsized":"2024-09-19T21:35:12.362178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:21.400292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"53'320","reported_seq":2751,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.056739+0000","last_change":"2024-09-19T21:34:32.024762+0000","last_active":"2024-09-19T21:35:18.056739+0000","last_peered":"2024-09-19T21:35:18.056739+0000","last_clean":"2024-09-19T21:35:18.056739+0000","last_became_active":"2024-09-19T21:34:30.006903+0000","last_became_peered":"2024-09-19T21:34:30.006903+0000","last_unstale":"2024-09-19T21:35:18.056739+0000","last_undegraded":"2024-09-19T21:35:18.056739+0000","last_fullsized":"2024-09-19T21:35:18.056739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.226419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":34830920,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":4213,"num_read_kb":7517,"num_write":344,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"53'410","reported_seq":3181,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.020874+0000","last_change":"2024-09-19T21:34:32.006802+0000","last_active":"2024-09-19T21:35:18.020874+0000","last_peered":"2024-09-19T21:35:18.020874+0000","last_clean":"2024-09-19T21:35:18.020874+0000","last_became_active":"2024-09-19T21:34:30.011547+0000","last_became_peered":"2024-09-19T21:34:30.011547+0000","last_unstale":"2024-09-19T21:35:18.020874+0000","last_undegraded":"2024-09-19T21:35:18.020874+0000","last_fullsized":"2024-09-19T21:35:18.020874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:45:47.281981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":2783318,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":6334,"num_read_kb":6481,"num_write":378,"num_write_kb":1759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":50,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.422235+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:35:12.422235+0000","last_peered":"2024-09-19T21:35:12.422235+0000","last_clean":"2024-09-19T21:35:12.422235+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:35:12.422235+0000","last_undegraded":"2024-09-19T21:35:12.422235+0000","last_fullsized":"2024-09-19T21:35:12.422235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"53'71","reported_seq":1619,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.996738+0000","last_change":"2024-09-19T21:34:32.011746+0000","last_active":"2024-09-19T21:35:17.996738+0000","last_peered":"2024-09-19T21:35:17.996738+0000","last_clean":"2024-09-19T21:35:17.996738+0000","last_became_active":"2024-09-19T21:34:30.013347+0000","last_became_peered":"2024-09-19T21:34:30.013347+0000","last_unstale":"2024-09-19T21:35:17.996738+0000","last_undegraded":"2024-09-19T21:35:17.996738+0000","last_fullsized":"2024-09-19T21:35:17.996738+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.129809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":21651012,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1563,"num_read_kb":27977,"num_write":84,"num_write_kb":3815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.361985+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:35:12.361985+0000","last_peered":"2024-09-19T21:35:12.361985+0000","last_clean":"2024-09-19T21:35:12.361985+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:35:12.361985+0000","last_undegraded":"2024-09-19T21:35:12.361985+0000","last_fullsized":"2024-09-19T21:35:12.361985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"53'142","reported_seq":2378,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.420307+0000","last_change":"2024-09-19T21:34:32.024303+0000","last_active":"2024-09-19T21:35:17.420307+0000","last_peered":"2024-09-19T21:35:17.420307+0000","last_clean":"2024-09-19T21:35:17.420307+0000","last_became_active":"2024-09-19T21:34:30.009730+0000","last_became_peered":"2024-09-19T21:34:30.009730+0000","last_unstale":"2024-09-19T21:35:17.420307+0000","last_undegraded":"2024-09-19T21:35:17.420307+0000","last_fullsized":"2024-09-19T21:35:17.420307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":43937280,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2900,"num_read_kb":6294,"num_write":109,"num_write_kb":1720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.360441+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:35:12.360441+0000","last_peered":"2024-09-19T21:35:12.360441+0000","last_clean":"2024-09-19T21:35:12.360441+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:35:12.360441+0000","last_undegraded":"2024-09-19T21:35:12.360441+0000","last_fullsized":"2024-09-19T21:35:12.360441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"53'228","reported_seq":6374,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:16.635536+0000","last_change":"2024-09-19T21:34:32.005130+0000","last_active":"2024-09-19T21:35:16.635536+0000","last_peered":"2024-09-19T21:35:16.635536+0000","last_clean":"2024-09-19T21:35:16.635536+0000","last_became_active":"2024-09-19T21:34:30.009669+0000","last_became_peered":"2024-09-19T21:34:30.009669+0000","last_unstale":"2024-09-19T21:35:16.635536+0000","last_undegraded":"2024-09-19T21:35:16.635536+0000","last_fullsized":"2024-09-19T21:35:16.635536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:20:42.350074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":73369759,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":7038,"num_read_kb":26476,"num_write":254,"num_write_kb":1560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":49,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.359386+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:35:12.359386+0000","last_peered":"2024-09-19T21:35:12.359386+0000","last_clean":"2024-09-19T21:35:12.359386+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:35:12.359386+0000","last_undegraded":"2024-09-19T21:35:12.359386+0000","last_fullsized":"2024-09-19T21:35:12.359386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":196,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-19T21:35:12.359043+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:35:12.359043+0000","last_peered":"2024-09-19T21:35:12.359043+0000","last_clean":"2024-09-19T21:35:12.359043+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:35:12.359043+0000","last_undegraded":"2024-09-19T21:35:12.359043+0000","last_fullsized":"2024-09-19T21:35:12.359043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"53'135","reported_seq":4948,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.080088+0000","last_change":"2024-09-19T21:34:32.024996+0000","last_active":"2024-09-19T21:35:18.080088+0000","last_peered":"2024-09-19T21:35:18.080088+0000","last_clean":"2024-09-19T21:35:18.080088+0000","last_became_active":"2024-09-19T21:34:30.009636+0000","last_became_peered":"2024-09-19T21:34:30.009636+0000","last_unstale":"2024-09-19T21:35:18.080088+0000","last_undegraded":"2024-09-19T21:35:18.080088+0000","last_fullsized":"2024-09-19T21:35:18.080088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:11.666768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":54877200,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":4632,"num_read_kb":24707,"num_write":118,"num_write_kb":2636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"53'108","reported_seq":1442,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.000365+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:35:18.000365+0000","last_peered":"2024-09-19T21:35:18.000365+0000","last_clean":"2024-09-19T21:35:18.000365+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:35:18.000365+0000","last_undegraded":"2024-09-19T21:35:18.000365+0000","last_fullsized":"2024-09-19T21:35:18.000365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":52108800,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1339,"num_read_kb":6950,"num_write":86,"num_write_kb":3323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"53'169","reported_seq":3687,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.077469+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:35:18.077469+0000","last_peered":"2024-09-19T21:35:18.077469+0000","last_clean":"2024-09-19T21:35:18.077469+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:35:18.077469+0000","last_undegraded":"2024-09-19T21:35:18.077469+0000","last_fullsized":"2024-09-19T21:35:18.077469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":12483162,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":4211,"num_read_kb":11081,"num_write":182,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"53'40","reported_seq":247,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:16.232592+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:35:16.232592+0000","last_peered":"2024-09-19T21:35:16.232592+0000","last_clean":"2024-09-19T21:35:16.232592+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:35:16.232592+0000","last_undegraded":"2024-09-19T21:35:16.232592+0000","last_fullsized":"2024-09-19T21:35:16.232592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":3480095,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":186,"num_read_kb":1848,"num_write":39,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"53'53","reported_seq":304,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.062003+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:35:18.062003+0000","last_peered":"2024-09-19T21:35:18.062003+0000","last_clean":"2024-09-19T21:35:18.062003+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:35:18.062003+0000","last_undegraded":"2024-09-19T21:35:18.062003+0000","last_fullsized":"2024-09-19T21:35:18.062003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":2560512,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":218,"num_read_kb":607,"num_write":52,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"53'122","reported_seq":4806,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.699890+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:35:17.699890+0000","last_peered":"2024-09-19T21:35:17.699890+0000","last_clean":"2024-09-19T21:35:17.699890+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:35:17.699890+0000","last_undegraded":"2024-09-19T21:35:17.699890+0000","last_fullsized":"2024-09-19T21:35:17.699890+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":48758784,"num_objects":28,"num_object_clones":11,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":4687,"num_read_kb":6561,"num_write":109,"num_write_kb":2316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"53'89","reported_seq":1693,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.019300+0000","last_change":"2024-09-19T21:34:32.024576+0000","last_active":"2024-09-19T21:35:18.019300+0000","last_peered":"2024-09-19T21:35:18.019300+0000","last_clean":"2024-09-19T21:35:18.019300+0000","last_became_active":"2024-09-19T21:34:30.006185+0000","last_became_peered":"2024-09-19T21:34:30.006185+0000","last_unstale":"2024-09-19T21:35:18.019300+0000","last_undegraded":"2024-09-19T21:35:18.019300+0000","last_fullsized":"2024-09-19T21:35:18.019300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:08:25.609727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":25352192,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1603,"num_read_kb":30777,"num_write":107,"num_write_kb":4040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"53'440","reported_seq":2404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.971640+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:35:17.971640+0000","last_peered":"2024-09-19T21:35:17.971640+0000","last_clean":"2024-09-19T21:35:17.971640+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:35:17.971640+0000","last_undegraded":"2024-09-19T21:35:17.971640+0000","last_fullsized":"2024-09-19T21:35:17.971640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":2217639,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":4821,"num_read_kb":5707,"num_write":565,"num_write_kb":851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"53'172","reported_seq":1921,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.077524+0000","last_change":"2024-09-19T21:34:32.000772+0000","last_active":"2024-09-19T21:35:18.077524+0000","last_peered":"2024-09-19T21:35:18.077524+0000","last_clean":"2024-09-19T21:35:18.077524+0000","last_became_active":"2024-09-19T21:34:30.007945+0000","last_became_peered":"2024-09-19T21:34:30.007945+0000","last_unstale":"2024-09-19T21:35:18.077524+0000","last_undegraded":"2024-09-19T21:35:18.077524+0000","last_fullsized":"2024-09-19T21:35:18.077524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:20:07.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":17158144,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":2650,"num_read_kb":6336,"num_write":128,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"53'135","reported_seq":1763,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.077598+0000","last_change":"2024-09-19T21:34:32.023713+0000","last_active":"2024-09-19T21:35:18.077598+0000","last_peered":"2024-09-19T21:35:18.077598+0000","last_clean":"2024-09-19T21:35:18.077598+0000","last_became_active":"2024-09-19T21:34:30.003969+0000","last_became_peered":"2024-09-19T21:34:30.003969+0000","last_unstale":"2024-09-19T21:35:18.077598+0000","last_undegraded":"2024-09-19T21:35:18.077598+0000","last_fullsized":"2024-09-19T21:35:18.077598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:36.748615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":48951332,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":2310,"num_read_kb":6935,"num_write":101,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"53'106","reported_seq":4412,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:18.077929+0000","last_change":"2024-09-19T21:34:32.011754+0000","last_active":"2024-09-19T21:35:18.077929+0000","last_peered":"2024-09-19T21:35:18.077929+0000","last_clean":"2024-09-19T21:35:18.077929+0000","last_became_active":"2024-09-19T21:34:30.012126+0000","last_became_peered":"2024-09-19T21:34:30.012126+0000","last_unstale":"2024-09-19T21:35:18.077929+0000","last_undegraded":"2024-09-19T21:35:18.077929+0000","last_fullsized":"2024-09-19T21:35:18.077929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:32.355336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":55559168,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":4332,"num_read_kb":50658,"num_write":117,"num_write_kb":5096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"53'155","reported_seq":4276,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T21:35:17.960030+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:35:17.960030+0000","last_peered":"2024-09-19T21:35:17.960030+0000","last_clean":"2024-09-19T21:35:17.960030+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:35:17.960030+0000","last_undegraded":"2024-09-19T21:35:17.960030+0000","last_fullsized":"2024-09-19T21:35:17.960030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":27912832,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":4296,"num_read_kb":8915,"num_write":145,"num_write_kb":2704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":996183400,"num_objects":641,"num_object_clones":174,"num_object_copies":1282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":202,"num_read":120781,"num_read_kb":425050,"num_write":6161,"num_write_kb":91437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117837824,"data_stored":116948176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5733,"ondisk_log_size":5733,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59268,"kb_used_data":24008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94312572,"statfs":{"total":96636764160,"available":96576073728,"internally_reserved":0,"allocated":24584192,"data_stored":24032678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.034}]},{"osd":4,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.93200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45288,"kb_used_data":10068,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94326552,"statfs":{"total":96636764160,"available":96590389248,"internally_reserved":0,"allocated":10309632,"data_stored":9796122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,47,0,54,90,53],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.373},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56116,"kb_used_data":20908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94315724,"statfs":{"total":96636764160,"available":96579301376,"internally_reserved":0,"allocated":21409792,"data_stored":20835267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61480,"kb_used_data":26272,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94310360,"statfs":{"total":96636764160,"available":96573808640,"internally_reserved":0,"allocated":26902528,"data_stored":26316603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.623}]},{"osd":4,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63108,"kb_used_data":23804,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94308732,"statfs":{"total":96636764160,"available":96572141568,"internally_reserved":0,"allocated":24375296,"data_stored":23978239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.183},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.31}]},{"osd":2,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.84399999999999997}]}]},{"osd":3,"up_from":12,"seq":51539607568,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49192,"kb_used_data":13984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94322648,"statfs":{"total":96636764160,"available":96586391552,"internally_reserved":0,"allocated":14319616,"data_stored":13920915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.6910000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.871}]},{"osd":2,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.5489999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":23781376,"data_stored":23747616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25718784,"data_stored":25495631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20836352,"data_stored":20624549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13766656,"data_stored":13717447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10264576,"data_stored":10101051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":23470080,"data_stored":23261882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:35:21.672 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [1, 0, 3, 5] 2024-09-19T21:35:21.673 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.15', '1', '0', '3', '5'] 2024-09-19T21:35:21.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.15 1 0 3 5 2024-09-19T21:35:21.680 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.680 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:21.730 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:21.726+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:21.796 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.796 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:21.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:21.847 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:21.846+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:21.912 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:21.912 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:21.914 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:35:21.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:21.932 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:21.925+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:21.970 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:21.991 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:22.020 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:22.026 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:22.026 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:22.033 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:22.029+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:22.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:22.043 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:22.071 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:22.079 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone16 from 0x8982400 (overlap 0x193de00) to 0xa18c00 2024-09-19T21:35:22.133 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:22.130+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:22.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 26.447795795000001, 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:22.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38478 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3237888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6110943720000002, 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050662393999999999, 2024-09-19T21:35:22.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38478 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585326+0000", 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.36e-06 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585339+0000", 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2836999999999999e-05 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585349+0000", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0227999999999999e-05 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635814+0000", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050465015000000002 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635883+0000", 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8490000000000003e-05 2024-09-19T21:35:22.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635986+0000", 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102972 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38479 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.212 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.6108852919999999, 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050617212000000002, 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38479 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585354+0000", 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998221 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585538+0000", 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000184007 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585548+0000", 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-19T21:35:22.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585555+0000", 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1380000000000001e-06 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636018+0000", 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050463348999999998 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636091+0000", 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3081e-05 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636150+0000", 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8387000000000002e-05 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38745 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3299328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5941315359999999, 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050507966000000001, 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38745 2024-09-19T21:35:22.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602096+0000", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998102 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602292+0000", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019607700000000001 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602303+0000", 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0248000000000001e-05 2024-09-19T21:35:22.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602320+0000", 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7898e-05 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652572+0000", 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050251704000000001 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652684+0000", 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011210299999999999 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652794+0000", 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109832 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.222 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38746 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3313664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5939564129999999, 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050552997000000002, 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38746 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602337+0000", 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999876 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602467+0000", 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000129827 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602478+0000", 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.057e-05 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602503+0000", 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5826e-05 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652823+0000", 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050319567000000003 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652916+0000", 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2750999999999993e-05 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653014+0000", 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8435000000000002e-05 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38747 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3328000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.227 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5938570160000003, 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050666628999999998, 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38747 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602511+0000", 2024-09-19T21:35:22.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999504 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602566+0000", 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4653e-05 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602574+0000", 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1799999999999996e-06 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602581+0000", 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653042+0000", 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050461113000000002 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653135+0000", 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2662000000000002e-05 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653227+0000", 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2315e-05 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.232 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38748 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5936691610000002, 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050685503999999999, 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38748 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602587+0000", 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998384 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602755+0000", 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000167875 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602781+0000", 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6815e-05 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602800+0000", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8644999999999999e-05 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653253+0000", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050452664000000001 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653338+0000", 2024-09-19T21:35:22.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5718999999999997e-05 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653434+0000", 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5524000000000002e-05 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38749 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3356672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.237 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5933966880000003, 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050628085000000003, 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38749 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602862+0000", 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998407 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603026+0000", 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016469600000000001 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603037+0000", 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0208e-05 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603049+0000", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2133999999999999e-05 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653466+0000", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050417464000000002 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653553+0000", 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6384999999999994e-05 2024-09-19T21:35:22.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653649+0000", 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6376e-05 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38750 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3371008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5931097640000003, 2024-09-19T21:35:22.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050551984000000001, 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38750 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603056+0000", 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997478 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603313+0000", 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025696099999999999 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603333+0000", 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0117000000000001e-05 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603341+0000", 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653674+0000", 2024-09-19T21:35:22.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050333152999999999 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653765+0000", 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1237999999999996e-05 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653860+0000", 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4347999999999999e-05 2024-09-19T21:35:22.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38751 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3385344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5928881590000001, 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050533023000000003, 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.248 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38751 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603350+0000", 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998207 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603534+0000", 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018449199999999999 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603544+0000", 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7489999999999993e-06 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603552+0000", 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8269999999999998e-06 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653888+0000", 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050335574000000001 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653975+0000", 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7108999999999996e-05 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654062+0000", 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7756000000000005e-05 2024-09-19T21:35:22.252 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38754 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3428352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5923073749999999, 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050586417000000002, 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38754 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604109+0000", 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604114+0000", 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604121+0000", 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7170000000000008e-06 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604127+0000", 2024-09-19T21:35:22.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9889999999999997e-06 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654520+0000", 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050392827000000001 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654605+0000", 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5191999999999993e-05 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654697+0000", 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1293999999999997e-05 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38755 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3442688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:22.258 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5921529909999999, 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050639968, 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38755 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604132+0000", 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998674 2024-09-19T21:35:22.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604276+0000", 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014431399999999999 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604295+0000", 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8457999999999998e-05 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604303+0000", 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9820000000000008e-06 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654725+0000", 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050422012000000002 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654811+0000", 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5921000000000006e-05 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654905+0000", 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3906000000000005e-05 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38775 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3729408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5879865149999999, 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050773897999999998, 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38775 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608432+0000", 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.424e-06 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608435+0000", 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608438+0000", 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.774e-06 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.658966+0000", 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050528196999999997 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659086+0000", 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119994 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659205+0000", 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118849 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38789 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3930112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5842414429999998, 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050834478000000002, 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38789 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.269 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612175+0000", 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612178+0000", 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9519999999999999e-06 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612185+0000", 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2740000000000004e-06 2024-09-19T21:35:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612192+0000", 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662685+0000", 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050493581000000003 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662845+0000", 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016009299999999999 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663011+0000", 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016543800000000001 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38790 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3944448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5842099090000001, 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051021812, 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.274 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38790 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612207+0000", 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612210+0000", 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000001e-06 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612217+0000", 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3159999999999999e-06 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612223+0000", 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663043+0000", 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050820585000000001 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663124+0000", 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0409999999999998e-05 2024-09-19T21:35:22.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663230+0000", 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105659 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38791 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3958784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5841900779999998, 2024-09-19T21:35:22.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051185702, 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38791 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:22.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612227+0000", 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612238+0000", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1326e-05 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612244+0000", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1430000000000003e-06 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612250+0000", 2024-09-19T21:35:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5249999999999996e-06 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663257+0000", 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051007515000000003 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663335+0000", 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7265e-05 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663413+0000", 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.852e-05 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38804 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4145152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5804477820000002, 2024-09-19T21:35:22.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050530841, 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38804 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:22.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615969+0000", 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615973+0000", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5209999999999998e-06 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615981+0000", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5269999999999998e-06 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615987+0000", 2024-09-19T21:35:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666333+0000", 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050346683000000003 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666413+0000", 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9437000000000005e-05 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666501+0000", 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7766e-05 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38805 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:22.289 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5804262720000004, 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050694516000000002, 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38805 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615994+0000", 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616006+0000", 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2201999999999999e-05 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616015+0000", 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4730000000000006e-06 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666526+0000", 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050510477999999998 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666598+0000", 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1769999999999999e-05 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666686+0000", 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8330000000000003e-05 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38806 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4173824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5803972819999998, 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0508669, 2024-09-19T21:35:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38806 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:22.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616023+0000", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2260000000000001e-06 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616031+0000", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8760000000000005e-06 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616036+0000", 2024-09-19T21:35:22.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.848e-06 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666710+0000", 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050674286999999998 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666789+0000", 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8916000000000001e-05 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666887+0000", 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8210999999999999e-05 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38807 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4188160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.5803774669999999, 2024-09-19T21:35:22.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051052267999999998, 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38807 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:22.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616042+0000", 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620000000000001e-06 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616048+0000", 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0310000000000004e-06 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616059+0000", 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0855e-05 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666927+0000", 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050868501000000003 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:22.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667017+0000", 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9625999999999999e-05 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667092+0000", 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5337000000000001e-05 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:22.307 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:22.301+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:22.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:22.407 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:22.402+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:22.426 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:22.426 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:22.426 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:22.427 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:22.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:22.509 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:22.506+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:22.545 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:22.545 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:22.546 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:22.546 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:22.546 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:22.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:22.609 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:22.606+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:22.672 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13312 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9150464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1599378460000001, 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039742026999999999, 2024-09-19T21:35:22.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13312 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508309+0000", 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508313+0000", 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508327+0000", 2024-09-19T21:35:22.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4273e-05 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508335+0000", 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3019999999999995e-06 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.547868+0000", 2024-09-19T21:35:22.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039532629999999999 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.547963+0000", 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5767000000000007e-05 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548052+0000", 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8449999999999995e-05 2024-09-19T21:35:22.678 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13313 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9164800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1599076440000005, 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039823512999999998, 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.679 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13313 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508342+0000", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6800000000000002e-06 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508351+0000", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6999999999999997e-06 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508356+0000", 2024-09-19T21:35:22.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9150000000000004e-06 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548075+0000", 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039719118999999997 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548119+0000", 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3621999999999997e-05 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548164+0000", 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4876999999999997e-05 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13314 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9179136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:22.684 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1598866369999996, 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040004148000000003, 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13314 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508363+0000", 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.588e-06 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508372+0000", 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7239999999999998e-06 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508378+0000", 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5160000000000002e-06 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548194+0000", 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039816161000000003 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548281+0000", 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7244999999999996e-05 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548365+0000", 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4288000000000003e-05 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13315 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9193472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1598657079999999, 2024-09-19T21:35:22.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040113412000000001, 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13315 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:22.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508384+0000", 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8009999999999999e-06 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508393+0000", 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3899999999999993e-06 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508398+0000", 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0370000000000002e-06 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548401+0000", 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040002807000000001 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548448+0000", 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7369e-05 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548495+0000", 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7355999999999999e-05 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13316 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9207808~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1598452980000005, 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040306070999999999, 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13316 2024-09-19T21:35:22.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508405+0000", 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508413+0000", 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4770000000000003e-06 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508418+0000", 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7219999999999999e-06 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548546+0000", 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040128108000000003 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548633+0000", 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7101999999999994e-05 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548709+0000", 2024-09-19T21:35:22.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5201e-05 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13317 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9222144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1598253409999995, 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040419458999999998, 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13317 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508429+0000", 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508438+0000", 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0219999999999993e-06 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508443+0000", 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.339e-06 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548742+0000", 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040298790000000001 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548792+0000", 2024-09-19T21:35:22.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.939e-05 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548842+0000", 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0274e-05 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13318 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9236480~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.705 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1597998839999999, 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040588994000000003, 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13318 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508447+0000", 2024-09-19T21:35:22.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508450+0000", 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7949999999999999e-06 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508459+0000", 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8829999999999995e-06 2024-09-19T21:35:22.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508464+0000", 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7380000000000002e-06 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548874+0000", 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040410089000000003 2024-09-19T21:35:22.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548964+0000", 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9979e-05 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549037+0000", 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2905999999999996e-05 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.710 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13319 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9250816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1597796440000003, 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040681254999999999, 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13319 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508470+0000", 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.554e-06 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508476+0000", 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4609999999999999e-06 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508483+0000", 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0899999999999999e-06 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549061+0000", 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040578102999999997 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549103+0000", 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1851999999999998e-05 2024-09-19T21:35:22.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549149+0000", 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6542000000000003e-05 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13320 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9265152~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.716 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1597607990000007, 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040884265000000003, 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13320 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508489+0000", 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5229999999999998e-06 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508498+0000", 2024-09-19T21:35:22.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7900000000000005e-06 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508503+0000", 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9749999999999998e-06 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549196+0000", 2024-09-19T21:35:22.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040692911999999998 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549292+0000", 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6574000000000001e-05 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549371+0000", 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8804e-05 2024-09-19T21:35:22.721 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13321 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9279488~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1597401739999995, 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040973554000000002, 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.722 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13321 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508507+0000", 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508510+0000", 2024-09-19T21:35:22.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8100000000000002e-06 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508519+0000", 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7860000000000002e-06 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508524+0000", 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9069999999999998e-06 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549395+0000", 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040871694 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549438+0000", 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2567000000000001e-05 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549481+0000", 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3313999999999999e-05 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13322 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9293824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1597185690000007, 2024-09-19T21:35:22.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041153020999999998, 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13322 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508531+0000", 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4820000000000001e-06 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508540+0000", 2024-09-19T21:35:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3210000000000005e-06 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508544+0000", 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7280000000000003e-06 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549503+0000", 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040958150999999998 2024-09-19T21:35:22.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549602+0000", 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9024000000000006e-05 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549682+0000", 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0648000000000004e-05 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.732 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13323 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9308160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1596993209999997, 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041246430000000001, 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13323 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508551+0000", 2024-09-19T21:35:22.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4880000000000001e-06 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508557+0000", 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1120000000000001e-06 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508562+0000", 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8099999999999997e-06 2024-09-19T21:35:22.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549709+0000", 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041147454 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549751+0000", 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2135000000000002e-05 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549795+0000", 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3752999999999999e-05 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13324 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9322496~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1596822400000004, 2024-09-19T21:35:22.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041451611999999999, 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13324 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508565+0000", 2024-09-19T21:35:22.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508567+0000", 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508576+0000", 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2859999999999999e-06 2024-09-19T21:35:22.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508581+0000", 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8760000000000004e-06 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549835+0000", 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041254836000000003 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549923+0000", 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7200999999999994e-05 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550017+0000", 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4534e-05 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13325 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9336832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.15966302, 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041551458999999999, 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13325 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508584+0000", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508587+0000", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5129999999999999e-06 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508595+0000", 2024-09-19T21:35:22.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3950000000000001e-06 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508603+0000", 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9189999999999999e-06 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550051+0000", 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041447770000000002 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550093+0000", 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1819000000000001e-05 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550136+0000", 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3356999999999998e-05 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13326 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9351168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1596394960000005, 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041737957999999999, 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13326 2024-09-19T21:35:22.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508611+0000", 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508619+0000", 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2490000000000001e-06 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508626+0000", 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.515e-06 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550181+0000", 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041554995999999997 2024-09-19T21:35:22.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550263+0000", 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1451000000000003e-05 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550346+0000", 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3436000000000005e-05 2024-09-19T21:35:22.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13327 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9365504~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1596164719999997, 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041838475999999999, 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13327 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:22.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508634+0000", 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6419999999999999e-06 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508642+0000", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3340000000000002e-06 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508647+0000", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8459999999999999e-06 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550383+0000", 2024-09-19T21:35:22.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041735815000000002 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550425+0000", 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2833000000000001e-05 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550470+0000", 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4365e-05 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13328 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9379840~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1595971089999999, 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042031341999999999, 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13328 2024-09-19T21:35:22.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508650+0000", 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508653+0000", 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1610000000000001e-06 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508660+0000", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8509999999999999e-06 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508665+0000", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.887e-06 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550524+0000", 2024-09-19T21:35:22.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041858575000000002 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550611+0000", 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7494e-05 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550682+0000", 2024-09-19T21:35:22.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13329 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9394176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1595779270000008, 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042148483, 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13329 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508669+0000", 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508676+0000", 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3239999999999999e-06 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508684+0000", 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6999999999999997e-06 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508689+0000", 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9479999999999999e-06 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550722+0000", 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042032928999999997 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550771+0000", 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9084e-05 2024-09-19T21:35:22.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550818+0000", 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7021000000000003e-05 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13330 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9408512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.769 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1595539049999992, 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042330257000000003, 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13330 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508696+0000", 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3e-06 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508704+0000", 2024-09-19T21:35:22.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3590000000000008e-06 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508709+0000", 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550859+0000", 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042149653000000002 2024-09-19T21:35:22.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550949+0000", 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0612000000000006e-05 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551024+0000", 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4723999999999998e-05 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13331 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9422848~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.1595309230000002, 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042420819999999998, 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:22.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13331 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508716+0000", 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508719+0000", 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5809999999999999e-06 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508727+0000", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4330000000000004e-06 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508733+0000", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440000000000002e-06 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551049+0000", 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042316486 2024-09-19T21:35:22.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551092+0000", 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2394000000000001e-05 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551138+0000", 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6134000000000002e-05 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:22.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:22.780 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:22.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:22.782 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:22.777+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:22.882 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:22.877+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:22.900 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:22.900 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:22.900 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:22.901 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:22.903 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-19T21:35:22.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:set 3.15 pg_upmap_items mapping to [1->0,3->5] 2024-09-19T21:35:22.984 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:22.982+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:23.017 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:23.017 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:23.017 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:23.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:23.085 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:23.081+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 67.193910867, 2024-09-19T21:35:23.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15154 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5314560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4842833790000007, 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056594848000000003, 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15154 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642372+0000", 2024-09-19T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999456 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642431+0000", 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8517000000000002e-05 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642439+0000", 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1980000000000001e-06 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642445+0000", 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3980000000000004e-06 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698816+0000", 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056370908999999997 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698937+0000", 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000120976 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699021+0000", 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4206e-05 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15155 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5328896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4841180509999994, 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056657103, 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15155 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642450+0000", 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998584 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642601+0000", 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015027000000000001 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642618+0000", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7482999999999998e-05 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642623+0000", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9409999999999998e-06 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699053+0000", 2024-09-19T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056430165999999997 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699162+0000", 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108626 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699249+0000", 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7064999999999994e-05 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15156 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5343232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4839966530000002, 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056741714999999998, 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15156 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642628+0000", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999146 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642716+0000", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8770000000000006e-05 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642722+0000", 2024-09-19T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6860000000000003e-06 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642726+0000", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9489999999999998e-06 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699283+0000", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556510999999997 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699378+0000", 2024-09-19T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5079000000000002e-05 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699455+0000", 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7245999999999999e-05 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15157 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5357568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4837155989999999, 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056672473000000001, 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15157 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642729+0000", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997349 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642999+0000", 2024-09-19T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026962200000000001 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643008+0000", 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6589999999999996e-06 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643014+0000", 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.888e-06 2024-09-19T21:35:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699488+0000", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056473795 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699590+0000", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010186100000000001 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699667+0000", 2024-09-19T21:35:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6627999999999999e-05 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15158 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5371904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4834738620000003, 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056666994999999998, 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15158 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643020+0000", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997845 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643241+0000", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022081999999999999 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643248+0000", 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0389999999999999e-06 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643255+0000", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7059999999999998e-06 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699701+0000", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056446488000000003 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699820+0000", 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119053 2024-09-19T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699903+0000", 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2595000000000005e-05 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15159 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5386240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4832782469999994, 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056686874999999998, 2024-09-19T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15159 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643259+0000", 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998279 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643435+0000", 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017585800000000001 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643442+0000", 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7889999999999997e-06 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643446+0000", 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4710000000000001e-06 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699931+0000", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056484178000000003 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700036+0000", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010582 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700119+0000", 2024-09-19T21:35:23.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2052999999999995e-05 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15160 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5400576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4831763569999996, 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056812712000000001, 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.166 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15160 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643450+0000", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999166 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643536+0000", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6331999999999999e-05 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643541+0000", 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5900000000000001e-06 2024-09-19T21:35:23.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643545+0000", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9049999999999999e-06 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700149+0000", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056604067000000001 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700262+0000", 2024-09-19T21:35:23.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113408 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700346+0000", 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3808999999999997e-05 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15161 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5414912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4829818550000002, 2024-09-19T21:35:23.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056828916, 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15161 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643548+0000", 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998202 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643731+0000", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018323300000000001 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643737+0000", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.088e-06 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643742+0000", 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.284e-06 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700374+0000", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056632688 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700481+0000", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106531 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700557+0000", 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5995999999999995e-05 2024-09-19T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15162 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5429248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4828615989999996, 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056920353, 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15162 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643745+0000", 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999897 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643852+0000", 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106959 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643862+0000", 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9529999999999994e-06 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643869+0000", 2024-09-19T21:35:23.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9199999999999998e-06 2024-09-19T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700585+0000", 2024-09-19T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056716492 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700685+0000", 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700769+0000", 2024-09-19T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3264999999999996e-05 2024-09-19T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15163 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5443584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4823530419999997, 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056623993999999997, 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15163 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.184 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643873+0000", 2024-09-19T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9995165 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644360+0000", 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048695100000000001 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644370+0000", 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6809999999999993e-06 2024-09-19T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644374+0000", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.639e-06 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700825+0000", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056450137999999997 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700915+0000", 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0080000000000005e-05 2024-09-19T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700981+0000", 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6202000000000001e-05 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15164 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5457920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4823313969999994, 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056756384, 2024-09-19T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15164 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644380+0000", 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9640000000000002e-06 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644384+0000", 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644388+0000", 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5040000000000004e-06 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701006+0000", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056617546999999997 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701079+0000", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3755000000000001e-05 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701135+0000", 2024-09-19T21:35:23.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448999999999997e-05 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15165 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5472256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4823176460000003, 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056848226000000002, 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.194 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15165 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:23.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644395+0000", 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9919999999999999e-06 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644401+0000", 2024-09-19T21:35:23.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2310000000000001e-06 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644406+0000", 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1429999999999997e-06 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701150+0000", 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056743891999999997 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701201+0000", 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0614e-05 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701240+0000", 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9697000000000003e-05 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15166 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5486592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.199 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.482193101, 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056825742999999998, 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15166 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644410+0000", 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998937 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644521+0000", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011017400000000001 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644526+0000", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644530+0000", 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:23.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701255+0000", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056724519000000001 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701292+0000", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7425000000000002e-05 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701343+0000", 2024-09-19T21:35:23.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0553999999999997e-05 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15167 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5500928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4819685279999995, 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056707884, 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15167 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644545+0000", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998035 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644746+0000", 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200779 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644756+0000", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0916e-05 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644761+0000", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.228e-06 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701357+0000", 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056595897999999999 2024-09-19T21:35:23.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701412+0000", 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5634000000000003e-05 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701449+0000", 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7017999999999997e-05 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15168 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5515264~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4817264170000009, 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056638200999999999, 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15168 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644775+0000", 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997921 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644987+0000", 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021153200000000001 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644999+0000", 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2089999999999999e-05 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645004+0000", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4020000000000004e-06 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701471+0000", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056467824999999999 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701560+0000", 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8195999999999994e-05 2024-09-19T21:35:23.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701622+0000", 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2130000000000003e-05 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15170 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5543936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4813470689999999, 2024-09-19T21:35:23.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056639429999999998, 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15170 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645362+0000", 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645365+0000", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0779999999999998e-06 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645368+0000", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6720000000000002e-06 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645371+0000", 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2480000000000001e-06 2024-09-19T21:35:23.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701839+0000", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056468061 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701937+0000", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7979999999999994e-05 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702002+0000", 2024-09-19T21:35:23.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4770000000000005e-05 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15171 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5558272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4810711990000005, 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556991000000001, 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15171 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645374+0000", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997354 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645642+0000", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026817799999999999 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645648+0000", 2024-09-19T21:35:23.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4389999999999997e-06 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645653+0000", 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1949999999999996e-06 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702027+0000", 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056374806 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702126+0000", 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8941e-05 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702196+0000", 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9239999999999994e-05 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.222 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15172 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5572608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4808765699999995, 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056523239000000003, 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15172 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645656+0000", 2024-09-19T21:35:23.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998231 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645837+0000", 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018102 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645845+0000", 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0649999999999999e-06 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645850+0000", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6619999999999996e-06 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702226+0000", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056376060999999998 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702303+0000", 2024-09-19T21:35:23.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7055e-05 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702357+0000", 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3517999999999998e-05 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15173 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5586944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:23.227 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.4807472419999996, 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056512898999999998, 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15173 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:23.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645853+0000", 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998908 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645966+0000", 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011243600000000001 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645972+0000", 2024-09-19T21:35:23.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.066e-06 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645977+0000", 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9320000000000004e-06 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702371+0000", 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394702999999997 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702428+0000", 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6508000000000002e-05 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702476+0000", 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7576e-05 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.233 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:23.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:23.235 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:23.230+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:23.336 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:23.334+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:23.352 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:23.352 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:23.352 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:23.363 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:23.437 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:23.433+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:23.473 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:23.473 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:23.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:23.538 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:23.534+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:23.596 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:23.596 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 42.364748278999997, 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:23.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:23.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:23.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890720471000002, 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:23.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:23.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890690643999999, 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:23.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890680958000001, 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890672632000001, 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.620 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:23.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:23.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890662240000001, 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:23.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.890651743999999, 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:23.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.888207655999999, 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:23.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:23.639 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.862990270000001, 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:23.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:23.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:23.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15066 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6164480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9673955789999997, 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029164998000000001, 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.646 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15066 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624793+0000", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624797+0000", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4580000000000002e-06 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624803+0000", 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6800000000000004e-06 2024-09-19T21:35:23.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624809+0000", 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8969999999999996e-06 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653805+0000", 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028996141999999999 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653889+0000", 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3906000000000006e-05 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653959+0000", 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9604000000000006e-05 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15067 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6178816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.650 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9673669700000005, 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029307910999999999, 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15067 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:23.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624831+0000", 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2019999999999993e-06 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624840+0000", 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5639999999999997e-06 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624846+0000", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4740000000000002e-06 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653998+0000", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029151795000000001 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654062+0000", 2024-09-19T21:35:23.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4289999999999996e-05 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654130+0000", 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8034000000000006e-05 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15068 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6193152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:23.655 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.967338969, 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029394144000000001, 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15068 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624853+0000", 2024-09-19T21:35:23.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8339999999999999e-06 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624860+0000", 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7689999999999999e-06 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624866+0000", 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0639999999999999e-06 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654143+0000", 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277609999999999 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654189+0000", 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5892e-05 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654245+0000", 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5383000000000002e-05 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15069 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9673186230000006, 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029542570000000001, 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15069 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624870+0000", 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624873+0000", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624880+0000", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624886+0000", 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:23.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654272+0000", 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029385674000000001 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654328+0000", 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6156999999999998e-05 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654413+0000", 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999998e-05 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15070 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.967298435, 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02965835, 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15070 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624894+0000", 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9900000000000002e-06 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624903+0000", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1220000000000001e-06 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624917+0000", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3973e-05 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654426+0000", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029508869 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654475+0000", 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9274999999999998e-05 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654549+0000", 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4492000000000004e-05 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15071 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6236160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:23.668 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9672664019999999, 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029922358, 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15071 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624922+0000", 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624926+0000", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2650000000000001e-06 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624933+0000", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624939+0000", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9819999999999996e-06 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654596+0000", 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029657246000000002 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654708+0000", 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112194 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654845+0000", 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013685999999999999 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15072 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6250496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9672449899999993, 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030057148999999998, 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15072 2024-09-19T21:35:23.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624947+0000", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.326e-06 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624954+0000", 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6590000000000001e-06 2024-09-19T21:35:23.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624959+0000", 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3630000000000002e-06 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654863+0000", 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029903855999999999 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654917+0000", 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4487000000000002e-05 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655002+0000", 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4300000000000003e-05 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15073 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6264832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:23.677 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9672239870000006, 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030212118999999999, 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15073 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:23.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624964+0000", 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624968+0000", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.16e-06 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624975+0000", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9600000000000003e-06 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624981+0000", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7980000000000002e-06 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655037+0000", 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030056058 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655093+0000", 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6223e-05 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655177+0000", 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4654000000000006e-05 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15074 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.9672037620000005, 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030446608, 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15074 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:23.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624985+0000", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624988+0000", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1370000000000002e-06 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625010+0000", 2024-09-19T21:35:23.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1778000000000001e-05 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625017+0000", 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5259999999999999e-06 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655204+0000", 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030186983000000001 2024-09-19T21:35:23.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655303+0000", 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655432+0000", 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012875899999999999 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18639 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 731136~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5702262910000009, 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029303412000000001, 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18639 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.688 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021964+0000", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021967+0000", 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:23.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021970+0000", 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2349999999999999e-06 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051070+0000", 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029100002 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051159+0000", 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9033e-05 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051266+0000", 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107069 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18640 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 745472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5702059209999994, 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029524404000000001, 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18640 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:23.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:23.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021985+0000", 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9750000000000001e-06 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021988+0000", 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9720000000000001e-06 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021991+0000", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6369999999999999e-06 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051317+0000", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029326609 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051398+0000", 2024-09-19T21:35:23.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0705999999999996e-05 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051508+0000", 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109948 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:23.697 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:23.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:23.699 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:23.693+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:23.800 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:23.798+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:23.820 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:23.820 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:23.820 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:23.830 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:23.901 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:23.898+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:23.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:35:23.947 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:23.947 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:23.947 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:23.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:24.002 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:23.998+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:24.066 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8095084339999996, 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:24.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094948950000003, 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:24.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:24.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.076 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094691550000004, 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094549410000003, 2024-09-19T21:35:24.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:24.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094340239999998, 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094186479999994, 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8094037200000006, 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8093511000000007, 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:24.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8093015220000002, 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.105 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:24.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8092843070000004, 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8092651689999997, 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:24.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8092503210000004, 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:24.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8092273280000004, 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:24.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:24.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8084950360000001, 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.127 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.131 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8084447259999994, 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8083824140000004, 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8070517810000002, 2024-09-19T21:35:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:24.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8069162030000001, 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:24.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8068394439999995, 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:24.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8067807200000008, 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:24.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:24.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:24.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:24.158 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:24.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:24.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:24.161 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:24.153+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:24.261 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:24.258+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:24.276 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:24.276 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:24.276 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:24.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:24.301 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:24.321 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:24.341 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:24.361 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:24.358+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:24.362 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:24.385 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:24.391 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:24.391 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:24.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:24.407 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:24.410 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:24.464 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:24.462+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:24.517 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:24.518 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:24.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:24.562 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:24.558+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:24.640 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:24.640 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:24.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:24.663 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:24.658+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:24.763 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:24.762+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:24.765 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:24.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:24.864 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:24.862+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:24.885 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:24.885 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:24.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:24.960 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:35:24.965 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:24.961+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:25.033 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.033 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:25.067 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:25.061+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:25.167 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:25.161+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:25.179 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.179 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:25.268 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:25.266+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:25.290 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.291 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:25.369 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:25.366+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:25.470 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:25.465+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:25.567 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.568 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.571 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:25.566+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:25.578 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:25.672 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:25.665+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:25.684 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.684 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:25.695 INFO:teuthology.orchestra.run.smithi122.stdout:2661 write 0x3e01400 thru 0x3e06f7a (0x5b7b bytes) 2024-09-19T21:35:25.738 INFO:teuthology.orchestra.run.smithi122.stdout:2662 write 0x91d5a00 thru 0x91dbf89 (0x658a bytes) 2024-09-19T21:35:25.741 INFO:teuthology.orchestra.run.smithi122.stdout:2663 read 0x3748600 thru 0x374c9c2 (0x43c3 bytes) 2024-09-19T21:35:25.772 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:25.769+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:25.801 INFO:teuthology.orchestra.run.smithi122.stdout:2665 write 0xd615c00 thru 0xd618e99 (0x329a bytes) 2024-09-19T21:35:25.802 INFO:teuthology.orchestra.run.smithi122.stdout:2666 punch from 0x80b3600 to 0x80b5e00, (0x2800 bytes) 2024-09-19T21:35:25.803 INFO:teuthology.orchestra.run.smithi122.stdout:2667 read 0x6478c00 thru 0x647fa0b (0x6e0c bytes) 2024-09-19T21:35:25.803 INFO:teuthology.orchestra.run.smithi122.stdout:2671 write 0x1d2d000 thru 0x1d2fa01 (0x2a02 bytes) 2024-09-19T21:35:25.804 INFO:teuthology.orchestra.run.smithi122.stdout:2672 punch from 0x81d6000 to 0x81d9c00, (0x3c00 bytes) 2024-09-19T21:35:25.805 INFO:teuthology.orchestra.run.smithi122.stdout:2675 write 0x3c3da00 thru 0x3c3f20a (0x180b bytes) 2024-09-19T21:35:25.811 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.811 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:25.836 INFO:teuthology.orchestra.run.smithi122.stdout:2676 trunc from 0xd619000 to 0x621d800 2024-09-19T21:35:25.872 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:25.870+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:25.897 INFO:teuthology.orchestra.run.smithi122.stdout:2677 write 0x3126a00 thru 0x3135914 (0xef15 bytes) 2024-09-19T21:35:25.904 INFO:teuthology.orchestra.run.smithi122.stdout:2678 trunc from 0x621d800 to 0x5caac00 2024-09-19T21:35:25.954 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:25.954 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:25.959 INFO:teuthology.orchestra.run.smithi122.stdout:2679 read 0x3dc8800 thru 0x3dd1c1c (0x941d bytes) 2024-09-19T21:35:25.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:25.974 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:25.969+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:25.995 INFO:teuthology.orchestra.run.smithi122.stdout:2681 write 0x9d49000 thru 0x9d55388 (0xc389 bytes) 2024-09-19T21:35:25.996 INFO:teuthology.orchestra.run.smithi122.stdout:2683 write 0x2877800 thru 0x2881437 (0x9c38 bytes) 2024-09-19T21:35:25.996 INFO:teuthology.orchestra.run.smithi122.stdout:2684 read 0x9964e00 thru 0x996c64e (0x784f bytes) 2024-09-19T21:35:25.996 INFO:teuthology.orchestra.run.smithi122.stdout:2686 punch from 0x6e81800 to 0x6e8f000, (0xd800 bytes) 2024-09-19T21:35:25.998 INFO:teuthology.orchestra.run.smithi122.stdout:2687 read 0x9779600 thru 0x977f79a (0x619b bytes) 2024-09-19T21:35:25.998 INFO:teuthology.orchestra.run.smithi122.stdout:2689 punch from 0x6515600 to 0x6522000, (0xca00 bytes) 2024-09-19T21:35:25.999 INFO:teuthology.orchestra.run.smithi122.stdout:2690 write 0x63f6000 thru 0x6403651 (0xd652 bytes) 2024-09-19T21:35:25.999 INFO:teuthology.orchestra.run.smithi122.stdout:2693 punch from 0x5016000 to 0x5018000, (0x2000 bytes) 2024-09-19T21:35:26.040 INFO:teuthology.orchestra.run.smithi122.stdout:2694 trunc from 0x9d55400 to 0x1787400 2024-09-19T21:35:26.074 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:26.069+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:26.114 INFO:teuthology.orchestra.run.smithi122.stdout:2695 read 0x13d200 thru 0x14445e (0x725f bytes) 2024-09-19T21:35:26.118 INFO:teuthology.orchestra.run.smithi122.stdout:2696 read 0x2dce00 thru 0x2dd78e (0x98f bytes) 2024-09-19T21:35:26.119 INFO:teuthology.orchestra.run.smithi122.stdout:2697 read 0x1ac800 thru 0x1ac80b (0xc bytes) 2024-09-19T21:35:26.132 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:26.132 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:26.140 INFO:teuthology.orchestra.run.smithi122.stdout:2699 trunc from 0x1787400 to 0xb710800 2024-09-19T21:35:26.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:26.158 INFO:teuthology.orchestra.run.smithi122.stdout:2702 write 0x7ccc800 thru 0x7cda3eb (0xdbec bytes) 2024-09-19T21:35:26.161 INFO:teuthology.orchestra.run.smithi122.stdout:2704 punch from 0x4fe9e00 to 0x4fef400, (0x5600 bytes) 2024-09-19T21:35:26.163 INFO:teuthology.orchestra.run.smithi122.stdout:2706 write 0xa71ea00 thru 0xa726771 (0x7d72 bytes) 2024-09-19T21:35:26.167 INFO:teuthology.orchestra.run.smithi122.stdout:2707 trunc from 0xb710800 to 0x9c78800 2024-09-19T21:35:26.175 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:26.170+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:26.209 INFO:teuthology.orchestra.run.smithi122.stdout:2708 write 0x734e000 thru 0x735ab81 (0xcb82 bytes) 2024-09-19T21:35:26.210 INFO:teuthology.orchestra.run.smithi122.stdout:2709 punch from 0x6045000 to 0x604dc00, (0x8c00 bytes) 2024-09-19T21:35:26.211 INFO:teuthology.orchestra.run.smithi122.stdout:2710 punch from 0x73c200 to 0x749a00, (0xd800 bytes) 2024-09-19T21:35:26.212 INFO:teuthology.orchestra.run.smithi122.stdout:2711 read 0x3bfa00 thru 0x3ca820 (0xae21 bytes) 2024-09-19T21:35:26.217 INFO:teuthology.orchestra.run.smithi122.stdout:2713 read 0x88e800 thru 0x893fa8 (0x57a9 bytes) 2024-09-19T21:35:26.219 INFO:teuthology.orchestra.run.smithi122.stdout:2714 read 0x453a000 thru 0x45400ae (0x60af bytes) 2024-09-19T21:35:26.219 INFO:teuthology.orchestra.run.smithi122.stdout:2715 write 0x729b200 thru 0x729e0fb (0x2efc bytes) 2024-09-19T21:35:26.235 INFO:teuthology.orchestra.run.smithi122.stdout:2718 trunc from 0x9c78800 to 0x1f36600 2024-09-19T21:35:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:26.270 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:26.276 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:26.273+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:26.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:26.320 INFO:teuthology.orchestra.run.smithi122.stdout:2723 write 0xbcb1600 thru 0xbcbb21a (0x9c1b bytes) 2024-09-19T21:35:26.321 INFO:teuthology.orchestra.run.smithi122.stdout:2725 read 0x7301000 thru 0x730fb1a (0xeb1b bytes) 2024-09-19T21:35:26.333 INFO:teuthology.orchestra.run.smithi122.stdout:2726 trunc from 0xbcbb400 to 0xe096800 2024-09-19T21:35:26.361 INFO:teuthology.orchestra.run.smithi122.stdout:2727 write 0xbb48e00 thru 0xbb51d78 (0x8f79 bytes) 2024-09-19T21:35:26.377 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:26.373+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:26.387 INFO:teuthology.orchestra.run.smithi122.stdout:2728 trunc from 0xe096800 to 0x1803600 2024-09-19T21:35:26.389 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:26.389 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:26.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:26.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:26.434 INFO:teuthology.orchestra.run.smithi122.stdout:2730 read 0xef0800 thru 0xefa112 (0x9913 bytes) 2024-09-19T21:35:26.447 INFO:teuthology.orchestra.run.smithi122.stdout:2735 trunc from 0x1803600 to 0x5b89000 2024-09-19T21:35:26.470 INFO:teuthology.orchestra.run.smithi122.stdout:2738 read 0x2334000 thru 0x234172a (0xd72b bytes) 2024-09-19T21:35:26.472 INFO:teuthology.orchestra.run.smithi122.stdout:2740 read 0x40f7000 thru 0x40ff5f1 (0x85f2 bytes) 2024-09-19T21:35:26.478 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:26.474+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:26.496 INFO:teuthology.orchestra.run.smithi122.stdout:2746 write 0x5bfc400 thru 0x5c0a6e1 (0xe2e2 bytes) 2024-09-19T21:35:26.503 INFO:teuthology.orchestra.run.smithi122.stdout:2748 trunc from 0x5c0a800 to 0x3be2800 2024-09-19T21:35:26.561 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:26.561 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:26.565 INFO:teuthology.orchestra.run.smithi122.stdout:2752 write 0x7d4f600 thru 0x7d53c8b (0x468c bytes) 2024-09-19T21:35:26.567 INFO:teuthology.orchestra.run.smithi122.stdout:2754 write 0xab0e00 thru 0xabc79c (0xb99d bytes) 2024-09-19T21:35:26.567 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:26.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:26.577 INFO:teuthology.orchestra.run.smithi122.stdout:2757 trunc from 0x7d53e00 to 0xc9b2a00 2024-09-19T21:35:26.578 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:26.574+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:26.591 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:26.606 INFO:teuthology.orchestra.run.smithi122.stdout:2758 read 0xc5ee00 thru 0xc5f9da (0xbdb bytes) 2024-09-19T21:35:26.607 INFO:teuthology.orchestra.run.smithi122.stdout:2759 read 0x381aa00 thru 0x3829113 (0xe714 bytes) 2024-09-19T21:35:26.609 INFO:teuthology.orchestra.run.smithi122.stdout:2760 trunc from 0xc9b2a00 to 0xc025600 2024-09-19T21:35:26.613 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:26.632 INFO:teuthology.orchestra.run.smithi122.stdout:2762 punch from 0xa2c0800 to 0xa2c3e00, (0x3600 bytes) 2024-09-19T21:35:26.643 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:2764 trunc from 0xc025600 to 0x186f600 2024-09-19T21:35:26.662 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:26.680 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:26.678+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:26.684 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:26.684 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:26.684 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:26.684 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:26.693 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:26.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:26.695 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:2765 punch from 0x4f6200 to 0x500c00, (0xaa00 bytes) 2024-09-19T21:35:26.735 INFO:teuthology.orchestra.run.smithi122.stdout:2767 write 0x283fe00 thru 0x28453be (0x55bf bytes) 2024-09-19T21:35:26.736 INFO:teuthology.orchestra.run.smithi122.stdout:2770 read 0x1d5ae00 thru 0x1d67dd7 (0xcfd8 bytes) 2024-09-19T21:35:26.736 INFO:teuthology.orchestra.run.smithi122.stdout:2771 read 0x24a3000 thru 0x24a943a (0x643b bytes) 2024-09-19T21:35:26.737 INFO:teuthology.orchestra.run.smithi122.stdout:2772 punch from 0x1a6c600 to 0x1a6f400, (0x2e00 bytes) 2024-09-19T21:35:26.753 INFO:teuthology.orchestra.run.smithi122.stdout:2773 trunc from 0x2845400 to 0xa1ce600 2024-09-19T21:35:26.781 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:26.776+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:26.800 INFO:teuthology.orchestra.run.smithi122.stdout:2774 write 0xbc3e000 thru 0xbc3f534 (0x1535 bytes) 2024-09-19T21:35:26.802 INFO:teuthology.orchestra.run.smithi122.stdout:2778 read 0x7c64000 thru 0x7c6cb29 (0x8b2a bytes) 2024-09-19T21:35:26.802 INFO:teuthology.orchestra.run.smithi122.stdout:2780 write 0x3b5f000 thru 0x3b60532 (0x1533 bytes) 2024-09-19T21:35:26.802 INFO:teuthology.orchestra.run.smithi122.stdout:2781 read 0x628be00 thru 0x6295c71 (0x9e72 bytes) 2024-09-19T21:35:26.808 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:26.809 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:26.809 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:26.812 INFO:teuthology.orchestra.run.smithi122.stdout:2782 trunc from 0xbc3f600 to 0xe436600 2024-09-19T21:35:26.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:26.849 INFO:teuthology.orchestra.run.smithi122.stdout:2783 write 0xd0cce00 thru 0xd0db990 (0xeb91 bytes) 2024-09-19T21:35:26.851 INFO:teuthology.orchestra.run.smithi122.stdout:2785 read 0x147de00 thru 0x14870cc (0x92cd bytes) 2024-09-19T21:35:26.852 INFO:teuthology.orchestra.run.smithi122.stdout:2786 punch from 0xb7e8800 to 0xb7f4200, (0xba00 bytes) 2024-09-19T21:35:26.855 INFO:teuthology.orchestra.run.smithi122.stdout:2787 trunc from 0xe436600 to 0xd685600 2024-09-19T21:35:26.881 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:26.876+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:26.894 INFO:teuthology.orchestra.run.smithi122.stdout:2788 write 0x26a6600 thru 0x26ae4a5 (0x7ea6 bytes) 2024-09-19T21:35:26.896 INFO:teuthology.orchestra.run.smithi122.stdout:2789 write 0xa52b000 thru 0xa530c8d (0x5c8e bytes) 2024-09-19T21:35:26.896 INFO:teuthology.orchestra.run.smithi122.stdout:2792 read 0x2132000 thru 0x2141dfb (0xfdfc bytes) 2024-09-19T21:35:26.900 INFO:teuthology.orchestra.run.smithi122.stdout:2799 trunc from 0xd685600 to 0xe4f8e00 2024-09-19T21:35:26.934 INFO:teuthology.orchestra.run.smithi122.stdout:2800 write 0xc8b3c00 thru 0xc8b4162 (0x563 bytes) 2024-09-19T21:35:26.936 INFO:teuthology.orchestra.run.smithi122.stdout:2801 read 0xc89f400 thru 0xc8a7200 (0x7e01 bytes) 2024-09-19T21:35:26.936 INFO:teuthology.orchestra.run.smithi122.stdout:2802 read 0xdc58800 thru 0xdc60723 (0x7f24 bytes) 2024-09-19T21:35:26.936 INFO:teuthology.orchestra.run.smithi122.stdout:2804 read 0x7ded800 thru 0x7deecaf (0x14b0 bytes) 2024-09-19T21:35:26.939 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1499 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2383872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.536120978, 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061259078000000002, 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1499 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400197+0000", 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400196+0000", 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400199+0000", 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9699999999999999e-06 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400219+0000", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9969000000000001e-05 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.400234+0000", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4545000000000001e-05 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461230+0000", 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060996214 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461338+0000", 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010734399999999999 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.461456+0000", 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118696 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1517 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.531967088999998, 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061368416000000002, 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1517 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404351+0000", 2024-09-19T21:35:26.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404353+0000", 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7e-06 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404367+0000", 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3567000000000001e-05 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404382+0000", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5457000000000001e-05 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465512+0000", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061130054000000003 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465613+0000", 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100515 2024-09-19T21:35:26.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465719+0000", 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010662699999999999 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1518 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.531929685999998, 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061530323999999997, 2024-09-19T21:35:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1518 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404388+0000", 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404391+0000", 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7300000000000001e-06 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404397+0000", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9900000000000002e-06 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404402+0000", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1200000000000001e-06 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465752+0000", 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061350564000000003 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465838+0000", 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5754999999999995e-05 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465919+0000", 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0741000000000005e-05 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.953 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4512.0:1519 3.8 3:114b27ad:::rbd_data.11068c3bf21f.0000000000000001:4 [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 49.531911856999997, 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061618757000000003, 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4512", 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3395645834", 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1519 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404406+0000", 2024-09-19T21:35:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404408+0000", 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404415+0000", 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0319999999999998e-06 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.404420+0000", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0799999999999996e-06 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465931+0000", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061510985999999997 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.465976+0000", 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4931000000000001e-05 2024-09-19T21:35:26.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:37.466025+0000", 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8828000000000001e-05 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9026 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13496320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.672065642, 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061312130999999999, 2024-09-19T21:35:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9026 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264252+0000", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264255+0000", 2024-09-19T21:35:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5220000000000002e-06 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264260+0000", 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2320000000000001e-06 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264265+0000", 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7280000000000003e-06 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325377+0000", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061111867 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325467+0000", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0564999999999998e-05 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325565+0000", 2024-09-19T21:35:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7547999999999995e-05 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9027 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13510656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.672015514, 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061437335000000003, 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9027 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264303+0000", 2024-09-19T21:35:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264302+0000", 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264306+0000", 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7850000000000002e-06 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264313+0000", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5320000000000001e-06 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.264319+0000", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6879999999999996e-06 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325612+0000", 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061292919000000001 2024-09-19T21:35:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325674+0000", 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2638999999999994e-05 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.325740+0000", 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5505000000000004e-05 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9390 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 13225984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.659924201000001, 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061207497999999999, 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9390 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276394+0000", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276348+0000", 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999542 2024-09-19T21:35:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276400+0000", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1529000000000001e-05 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276420+0000", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0203e-05 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.276428+0000", 2024-09-19T21:35:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9990000000000008e-06 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337400+0000", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060972560000000002 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337501+0000", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010045800000000001 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.337601+0000", 2024-09-19T21:35:26.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000100703 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9606 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12640256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.651775882000001, 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061298120999999997, 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9606 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284542+0000", 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284541+0000", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284547+0000", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6470000000000003e-06 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284566+0000", 2024-09-19T21:35:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8986000000000001e-05 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284578+0000", 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1578e-05 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345651+0000", 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061073167999999997 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345717+0000", 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6042000000000002e-05 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345840+0000", 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012361699999999999 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9607 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12654592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.651455441, 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061227252000000003, 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9607 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284863+0000", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284585+0000", 2024-09-19T21:35:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997225 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284868+0000", 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028231399999999998 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284890+0000", 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2062e-05 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284896+0000", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.246e-06 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345877+0000", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.060981568999999999 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.345976+0000", 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8826999999999994e-05 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346090+0000", 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113701 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9608 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12668928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.651326584, 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061372768000000001, 2024-09-19T21:35:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9608 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284992+0000", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284905+0000", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999132 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.284997+0000", 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2140999999999998e-05 2024-09-19T21:35:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285012+0000", 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4800999999999999e-05 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285032+0000", 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0514e-05 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346127+0000", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061094674000000002 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346223+0000", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5798000000000007e-05 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346364+0000", 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000141691 2024-09-19T21:35:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9609 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12683264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.651065959, 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061363407000000002, 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9609 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285252+0000", 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285039+0000", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997869 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285258+0000", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000219096 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285269+0000", 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.098e-05 2024-09-19T21:35:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285276+0000", 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5950000000000004e-06 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346393+0000", 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061116960999999997 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346495+0000", 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010218299999999999 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346616+0000", 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120654 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9610 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12697600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.650836576, 2024-09-19T21:35:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061447314000000003, 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9610 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285482+0000", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285280+0000", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997988 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285487+0000", 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020666400000000001 2024-09-19T21:35:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285498+0000", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1069e-05 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285506+0000", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7449999999999995e-06 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346663+0000", 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061157126999999999 2024-09-19T21:35:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346785+0000", 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012234600000000001 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346929+0000", 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014350699999999999 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9611 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12711936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.65057223, 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061431025, 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9611 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:26.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285746+0000", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285512+0000", 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997663 2024-09-19T21:35:26.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285751+0000", 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000238555 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285758+0000", 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6660000000000008e-06 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:26.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285764+0000", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4709999999999998e-06 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.346967+0000", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061203562000000003 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347064+0000", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6366000000000005e-05 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:26.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347177+0000", 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000113276 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9612 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12726272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.650338830000001, 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061441227000000001, 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:26.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9612 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285979+0000", 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285768+0000", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997888 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285984+0000", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021622699999999999 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.285995+0000", 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0631e-05 2024-09-19T21:35:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286011+0000", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5828000000000002e-05 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347226+0000", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061214888000000002 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347346+0000", 2024-09-19T21:35:27.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000120116 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347421+0000", 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4599999999999997e-05 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9613 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12740608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.650088777000001, 2024-09-19T21:35:27.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061446091000000001, 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9613 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286229+0000", 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286023+0000", 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999794 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286234+0000", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000210609 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286241+0000", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4710000000000001e-06 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286247+0000", 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4140000000000002e-06 2024-09-19T21:35:27.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347460+0000", 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061213103999999997 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347557+0000", 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7386999999999994e-05 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347675+0000", 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011809199999999999 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9614 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12754944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.649823265, 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061377640999999997, 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9614 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286495+0000", 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286257+0000", 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997625 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286501+0000", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00024361599999999999 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286511+0000", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0614e-05 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286526+0000", 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4594e-05 2024-09-19T21:35:27.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347724+0000", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061198065000000003 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347794+0000", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0146000000000002e-05 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347873+0000", 2024-09-19T21:35:27.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8280000000000003e-05 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9615 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12769280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.649582632, 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061393000000000003, 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9615 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286736+0000", 2024-09-19T21:35:27.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286534+0000", 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997988 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286742+0000", 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020750099999999999 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286754+0000", 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2123999999999999e-05 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286762+0000", 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.687e-06 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.347907+0000", 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061144153999999999 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348009+0000", 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010231 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348129+0000", 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119628 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9616 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12783616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.649305204999999, 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061308548999999997, 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9616 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287013+0000", 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.286767+0000", 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997544 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287019+0000", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025142099999999999 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287041+0000", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2878e-05 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287050+0000", 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.884e-06 2024-09-19T21:35:27.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348161+0000", 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061110730000000002 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348226+0000", 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4586999999999997e-05 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348321+0000", 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5791000000000005e-05 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9617 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12797952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.649116093, 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061353525999999999, 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9617 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287202+0000", 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287060+0000", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998579 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287208+0000", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014783 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287226+0000", 2024-09-19T21:35:27.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8695e-05 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287242+0000", 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5574e-05 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348348+0000", 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061105680000000002 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348447+0000", 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9481999999999994e-05 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348556+0000", 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108545 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:9618 3.4 3:207e76c2:::rbd_data.11066d5522c8.0000000000000005:12 [read 12812288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.026 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 12.648800848, 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061298677000000003, 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 9618 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287517+0000", 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287250+0000", 2024-09-19T21:35:27.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997325 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287524+0000", 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027381900000000002 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287538+0000", 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4313e-05 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.287547+0000", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9649999999999997e-06 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348597+0000", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.061049919000000001 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348676+0000", 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8997000000000003e-05 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.348816+0000", 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014009999999999999 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.031 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:27.031 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:35:27.033 INFO:teuthology.orchestra.run.smithi122.stdout:2805 trunc from 0xe4f8e00 to 0x341d600 2024-09-19T21:35:27.033 INFO:teuthology.orchestra.run.smithi122.stdout:2807 write 0x14bfe00 thru 0x14cc0e5 (0xc2e6 bytes) 2024-09-19T21:35:27.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:27.034 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:27.028+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:2809 write 0x9bcf400 thru 0x9bda62a (0xb22b bytes) 2024-09-19T21:35:27.066 INFO:teuthology.orchestra.run.smithi122.stdout:2810 read 0x27b2200 thru 0x27c11de (0xefdf bytes) 2024-09-19T21:35:27.067 INFO:teuthology.orchestra.run.smithi122.stdout:2813 read 0x8cdd000 thru 0x8ce1027 (0x4028 bytes) 2024-09-19T21:35:27.067 INFO:teuthology.orchestra.run.smithi122.stdout:2814 read 0x9a0c800 thru 0x9a18328 (0xbb29 bytes) 2024-09-19T21:35:27.067 INFO:teuthology.orchestra.run.smithi122.stdout:2815 write 0x1974000 thru 0x1981e06 (0xde07 bytes) 2024-09-19T21:35:27.103 INFO:teuthology.orchestra.run.smithi122.stdout:2818 write 0xa14d200 thru 0xa159e56 (0xcc57 bytes) 2024-09-19T21:35:27.104 INFO:teuthology.orchestra.run.smithi122.stdout:2820 read 0x7fd2e00 thru 0x7fdb539 (0x873a bytes) 2024-09-19T21:35:27.122 INFO:teuthology.orchestra.run.smithi122.stdout:2824 trunc from 0xa15a000 to 0x286ba00 2024-09-19T21:35:27.133 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:27.130+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:27.162 INFO:teuthology.orchestra.run.smithi122.stdout:2825 read 0x1acf800 thru 0x1ad4300 (0x4b01 bytes) 2024-09-19T21:35:27.169 INFO:teuthology.orchestra.run.smithi122.stdout:2826 trunc from 0x286ba00 to 0x1632200 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_repop(client.5154.0:3408 3.1a e58/23 3:5ccad9d7:::rbd_object_map.1106bae2d13f:head v 58'291, mlcod=58'289)", 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:27.200044+0000", 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.000599363, 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00063433900000000004, 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200044+0000", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200044+0000", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200043+0000", 2024-09-19T21:35:27.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200052+0000", 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3789999999999995e-06 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200075+0000", 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3351999999999999e-05 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200081+0000", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6969999999999998e-06 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200128+0000", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7302999999999998e-05 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:27.200153+0000", 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4417e-05 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-19T21:35:27.206 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 1 2024-09-19T21:35:27.206 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:27.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:27.223 INFO:teuthology.orchestra.run.smithi122.stdout:2827 read 0x254000 thru 0x255126 (0x1127 bytes) 2024-09-19T21:35:27.225 INFO:teuthology.orchestra.run.smithi122.stdout:2830 read 0xbc9000 thru 0xbd1266 (0x8267 bytes) 2024-09-19T21:35:27.227 INFO:teuthology.orchestra.run.smithi122.stdout:2831 clone 18 order 24 su 262144 sc 2 2024-09-19T21:35:27.234 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:27.228+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:27.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:27.334 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:27.328+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:27.432 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:27.432 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:27.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:27.432 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:27.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 31.679756671, 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:27.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:27.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38478 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3237888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8430552480000006, 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050662393999999999, 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.439 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38478 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:27.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585326+0000", 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.36e-06 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585339+0000", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2836999999999999e-05 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585349+0000", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0227999999999999e-05 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635814+0000", 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050465015000000002 2024-09-19T21:35:27.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635883+0000", 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8490000000000003e-05 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635986+0000", 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102972 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38479 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8428461679999995, 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050617212000000002, 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38479 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585354+0000", 2024-09-19T21:35:27.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998221 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585538+0000", 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000184007 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585548+0000", 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585555+0000", 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1380000000000001e-06 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636018+0000", 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050463348999999998 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636091+0000", 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3081e-05 2024-09-19T21:35:27.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636150+0000", 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8387000000000002e-05 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38745 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3299328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8260924119999995, 2024-09-19T21:35:27.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050507966000000001, 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38745 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602096+0000", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998102 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602292+0000", 2024-09-19T21:35:27.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019607700000000001 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602303+0000", 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0248000000000001e-05 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602320+0000", 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7898e-05 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652572+0000", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050251704000000001 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652684+0000", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011210299999999999 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652794+0000", 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109832 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.452 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38746 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3313664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8259172889999995, 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050552997000000002, 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38746 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602337+0000", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999876 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602467+0000", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000129827 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602478+0000", 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.057e-05 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602503+0000", 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5826e-05 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652823+0000", 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050319567000000003 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652916+0000", 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2750999999999993e-05 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653014+0000", 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8435000000000002e-05 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38747 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3328000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8258178919999999, 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050666628999999998, 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38747 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602511+0000", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999504 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602566+0000", 2024-09-19T21:35:27.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4653e-05 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602574+0000", 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1799999999999996e-06 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602581+0000", 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653042+0000", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050461113000000002 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653135+0000", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2662000000000002e-05 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653227+0000", 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2315e-05 2024-09-19T21:35:27.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38748 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8256300369999998, 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050685503999999999, 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38748 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602587+0000", 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998384 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602755+0000", 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000167875 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602781+0000", 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6815e-05 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602800+0000", 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8644999999999999e-05 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653253+0000", 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050452664000000001 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653338+0000", 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5718999999999997e-05 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653434+0000", 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5524000000000002e-05 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38749 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3356672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8253575640000008, 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050628085000000003, 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38749 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602862+0000", 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998407 2024-09-19T21:35:27.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603026+0000", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016469600000000001 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603037+0000", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0208e-05 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603049+0000", 2024-09-19T21:35:27.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2133999999999999e-05 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653466+0000", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050417464000000002 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653553+0000", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6384999999999994e-05 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653649+0000", 2024-09-19T21:35:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6376e-05 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38750 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3371008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8250706399999999, 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050551984000000001, 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38750 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603056+0000", 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997478 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603313+0000", 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025696099999999999 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603333+0000", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0117000000000001e-05 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603341+0000", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653674+0000", 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050333152999999999 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653765+0000", 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1237999999999996e-05 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653860+0000", 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4347999999999999e-05 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38751 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3385344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8248490349999997, 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050533023000000003, 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38751 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603350+0000", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998207 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603534+0000", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018449199999999999 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603544+0000", 2024-09-19T21:35:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7489999999999993e-06 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603552+0000", 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8269999999999998e-06 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653888+0000", 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050335574000000001 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653975+0000", 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7108999999999996e-05 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654062+0000", 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7756000000000005e-05 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38754 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3428352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8242682509999995, 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050586417000000002, 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38754 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604109+0000", 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:27.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604114+0000", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604121+0000", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7170000000000008e-06 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604127+0000", 2024-09-19T21:35:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9889999999999997e-06 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654520+0000", 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050392827000000001 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654605+0000", 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5191999999999993e-05 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654697+0000", 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1293999999999997e-05 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38755 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3442688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8241138669999994, 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050639968, 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38755 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604132+0000", 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998674 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604276+0000", 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014431399999999999 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604295+0000", 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8457999999999998e-05 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604303+0000", 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9820000000000008e-06 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654725+0000", 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050422012000000002 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654811+0000", 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5921000000000006e-05 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654905+0000", 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3906000000000005e-05 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38775 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3729408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8199473909999995, 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050773897999999998, 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38775 2024-09-19T21:35:27.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608432+0000", 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.424e-06 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608435+0000", 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:27.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608438+0000", 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.774e-06 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.658966+0000", 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050528196999999997 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659086+0000", 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119994 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659205+0000", 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118849 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38789 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3930112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8162023190000003, 2024-09-19T21:35:27.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050834478000000002, 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38789 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612175+0000", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612178+0000", 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9519999999999999e-06 2024-09-19T21:35:27.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612185+0000", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2740000000000004e-06 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612192+0000", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662685+0000", 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050493581000000003 2024-09-19T21:35:27.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662845+0000", 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016009299999999999 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663011+0000", 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016543800000000001 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38790 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3944448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8161707850000006, 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051021812, 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38790 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612207+0000", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612210+0000", 2024-09-19T21:35:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000001e-06 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612217+0000", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3159999999999999e-06 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612223+0000", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663043+0000", 2024-09-19T21:35:27.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050820585000000001 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663124+0000", 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0409999999999998e-05 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663230+0000", 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105659 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38791 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3958784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8161509539999994, 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051185702, 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38791 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612227+0000", 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612238+0000", 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1326e-05 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612244+0000", 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1430000000000003e-06 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612250+0000", 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5249999999999996e-06 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663257+0000", 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051007515000000003 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663335+0000", 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7265e-05 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663413+0000", 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.852e-05 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38804 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4145152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8124086580000007, 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050530841, 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38804 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615969+0000", 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615973+0000", 2024-09-19T21:35:27.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5209999999999998e-06 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615981+0000", 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5269999999999998e-06 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615987+0000", 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666333+0000", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050346683000000003 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666413+0000", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9437000000000005e-05 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666501+0000", 2024-09-19T21:35:27.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7766e-05 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38805 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8123871479999991, 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050694516000000002, 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38805 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615994+0000", 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616006+0000", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2201999999999999e-05 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616015+0000", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4730000000000006e-06 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666526+0000", 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050510477999999998 2024-09-19T21:35:27.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666598+0000", 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1769999999999999e-05 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666686+0000", 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8330000000000003e-05 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38806 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4173824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8123581580000003, 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0508669, 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38806 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.517 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616023+0000", 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2260000000000001e-06 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616031+0000", 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8760000000000005e-06 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616036+0000", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.848e-06 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666710+0000", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050674286999999998 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666789+0000", 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8916000000000001e-05 2024-09-19T21:35:27.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666887+0000", 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8210999999999999e-05 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38807 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4188160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.8123383430000004, 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051052267999999998, 2024-09-19T21:35:27.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38807 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616042+0000", 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620000000000001e-06 2024-09-19T21:35:27.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616048+0000", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0310000000000004e-06 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616059+0000", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0855e-05 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666927+0000", 2024-09-19T21:35:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050868501000000003 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667017+0000", 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9625999999999999e-05 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667092+0000", 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5337000000000001e-05 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:27.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:27.526 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:27.527 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:27.528 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:27.522+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:27.628 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:27.626+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:27.647 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:27.647 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:27.647 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:27.647 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:27.657 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:27.730 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:27.726+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:27.761 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:27.761 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:27.761 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:27.761 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:27.761 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:27.771 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:27.831 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:27.826+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:27.883 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:27.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.884 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13312 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9150464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.884 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:27.884 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369643023, 2024-09-19T21:35:27.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039742026999999999, 2024-09-19T21:35:27.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.886 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13312 2024-09-19T21:35:27.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.886 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:27.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508310+0000", 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508309+0000", 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508313+0000", 2024-09-19T21:35:27.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1640000000000001e-06 2024-09-19T21:35:27.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508327+0000", 2024-09-19T21:35:27.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4273e-05 2024-09-19T21:35:27.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508335+0000", 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3019999999999995e-06 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.547868+0000", 2024-09-19T21:35:27.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039532629999999999 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.547963+0000", 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5767000000000007e-05 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548052+0000", 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8449999999999995e-05 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13313 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9164800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369612821, 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039823512999999998, 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13313 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508340+0000", 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508342+0000", 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6800000000000002e-06 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508351+0000", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6999999999999997e-06 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508356+0000", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9150000000000004e-06 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548075+0000", 2024-09-19T21:35:27.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039719118999999997 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548119+0000", 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3621999999999997e-05 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548164+0000", 2024-09-19T21:35:27.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4876999999999997e-05 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13314 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9179136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369591814, 2024-09-19T21:35:27.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040004148000000003, 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13314 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508361+0000", 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508363+0000", 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.588e-06 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508372+0000", 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7239999999999998e-06 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508378+0000", 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5160000000000002e-06 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548194+0000", 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.039816161000000003 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548281+0000", 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7244999999999996e-05 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548365+0000", 2024-09-19T21:35:27.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4288000000000003e-05 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13315 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9193472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369570885, 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040113412000000001, 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13315 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508382+0000", 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508384+0000", 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8009999999999999e-06 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508393+0000", 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3899999999999993e-06 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508398+0000", 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0370000000000002e-06 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548401+0000", 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040002807000000001 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548448+0000", 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7369e-05 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548495+0000", 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7355999999999999e-05 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13316 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9207808~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369550475, 2024-09-19T21:35:27.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040306070999999999, 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13316 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508402+0000", 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508405+0000", 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8370000000000001e-06 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508413+0000", 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4770000000000003e-06 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508418+0000", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7219999999999999e-06 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548546+0000", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040128108000000003 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548633+0000", 2024-09-19T21:35:27.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7101999999999994e-05 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548709+0000", 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5201e-05 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13317 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9222144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:27.918 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369530517999999, 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040419458999999998, 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13317 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:27.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508422+0000", 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508429+0000", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508438+0000", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0219999999999993e-06 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508443+0000", 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.339e-06 2024-09-19T21:35:27.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548742+0000", 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040298790000000001 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548792+0000", 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.939e-05 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548842+0000", 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0274e-05 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13318 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9236480~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369505061, 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040588994000000003, 2024-09-19T21:35:27.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13318 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508448+0000", 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508447+0000", 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508450+0000", 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7949999999999999e-06 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508459+0000", 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8829999999999995e-06 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508464+0000", 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7380000000000002e-06 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548874+0000", 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040410089000000003 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.548964+0000", 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9979e-05 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549037+0000", 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2905999999999996e-05 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13319 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9250816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369484821, 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040681254999999999, 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13319 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508468+0000", 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508470+0000", 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.554e-06 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508476+0000", 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4609999999999999e-06 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508483+0000", 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0899999999999999e-06 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549061+0000", 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040578102999999997 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549103+0000", 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1851999999999998e-05 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549149+0000", 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6542000000000003e-05 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13320 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9265152~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369465976000001, 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040884265000000003, 2024-09-19T21:35:27.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13320 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:27.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508487+0000", 2024-09-19T21:35:27.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508489+0000", 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5229999999999998e-06 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508498+0000", 2024-09-19T21:35:27.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7900000000000005e-06 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508503+0000", 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9749999999999998e-06 2024-09-19T21:35:27.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549196+0000", 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040692911999999998 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549292+0000", 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6574000000000001e-05 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549371+0000", 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8804e-05 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13321 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9279488~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369445351, 2024-09-19T21:35:27.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040973554000000002, 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13321 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508508+0000", 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508507+0000", 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508510+0000", 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8100000000000002e-06 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508519+0000", 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7860000000000002e-06 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508524+0000", 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9069999999999998e-06 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549395+0000", 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040871694 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549438+0000", 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2567000000000001e-05 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549481+0000", 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3313999999999999e-05 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13322 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9293824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369423746000001, 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041153020999999998, 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13322 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508529+0000", 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508531+0000", 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4820000000000001e-06 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508540+0000", 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3210000000000005e-06 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508544+0000", 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7280000000000003e-06 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549503+0000", 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.040958150999999998 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549602+0000", 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9024000000000006e-05 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549682+0000", 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0648000000000004e-05 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13323 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9308160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369404498, 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041246430000000001, 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13323 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508548+0000", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508551+0000", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4880000000000001e-06 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508557+0000", 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1120000000000001e-06 2024-09-19T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508562+0000", 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8099999999999997e-06 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549709+0000", 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041147454 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549751+0000", 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2135000000000002e-05 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549795+0000", 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3752999999999999e-05 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13324 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9322496~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369387417, 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041451611999999999, 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13324 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508566+0000", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508565+0000", 2024-09-19T21:35:27.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508567+0000", 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2759999999999999e-06 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508576+0000", 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2859999999999999e-06 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508581+0000", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8760000000000004e-06 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549835+0000", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041254836000000003 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.549923+0000", 2024-09-19T21:35:27.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7200999999999994e-05 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550017+0000", 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4534e-05 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13325 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9336832~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:27.961 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369368197, 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041551458999999999, 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13325 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508585+0000", 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508584+0000", 2024-09-19T21:35:27.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508587+0000", 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5129999999999999e-06 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508595+0000", 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3950000000000001e-06 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508603+0000", 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9189999999999999e-06 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550051+0000", 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041447770000000002 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550093+0000", 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1819000000000001e-05 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550136+0000", 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3356999999999998e-05 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13326 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9351168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369344673000001, 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041737957999999999, 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13326 2024-09-19T21:35:27.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508608+0000", 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508611+0000", 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6299999999999998e-06 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508619+0000", 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2490000000000001e-06 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508626+0000", 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.515e-06 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550181+0000", 2024-09-19T21:35:27.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041554995999999997 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550263+0000", 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1451000000000003e-05 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550346+0000", 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3436000000000005e-05 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.971 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13327 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9365504~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369321649, 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041838475999999999, 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13327 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508631+0000", 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508634+0000", 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6419999999999999e-06 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508642+0000", 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3340000000000002e-06 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508647+0000", 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8459999999999999e-06 2024-09-19T21:35:27.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550383+0000", 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041735815000000002 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550425+0000", 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2833000000000001e-05 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550470+0000", 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4365e-05 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13328 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9379840~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369302286, 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042031341999999999, 2024-09-19T21:35:27.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13328 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508651+0000", 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508650+0000", 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508653+0000", 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1610000000000001e-06 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508660+0000", 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8509999999999999e-06 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508665+0000", 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.887e-06 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550524+0000", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.041858575000000002 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550611+0000", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7494e-05 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550682+0000", 2024-09-19T21:35:27.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0662999999999998e-05 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13329 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9394176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369283104000001, 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042148483, 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.982 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13329 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508670+0000", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508669+0000", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508676+0000", 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3239999999999999e-06 2024-09-19T21:35:27.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508684+0000", 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6999999999999997e-06 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508689+0000", 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9479999999999999e-06 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550722+0000", 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042032928999999997 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550771+0000", 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9084e-05 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550818+0000", 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7021000000000003e-05 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.986 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13330 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9408512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.369259081999999, 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042330257000000003, 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13330 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508694+0000", 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508696+0000", 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3e-06 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508704+0000", 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3590000000000008e-06 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508709+0000", 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550859+0000", 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042149653000000002 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.550949+0000", 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0612000000000006e-05 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551024+0000", 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4723999999999998e-05 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:13331 3.15 3:adabfd7f:::rbd_data.11066d5522c8.0000000000000009:12 [read 9422848~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.3692361, 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042420819999999998, 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 13331 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:27.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508717+0000", 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508716+0000", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508719+0000", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5809999999999999e-06 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508727+0000", 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4330000000000004e-06 2024-09-19T21:35:27.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.508733+0000", 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1440000000000002e-06 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551049+0000", 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.042316486 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551092+0000", 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2394000000000001e-05 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.551138+0000", 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6134000000000002e-05 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:27.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:27.997 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:27.997 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:27.997 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:27.997 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T21:35:27.998 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:35:27.999 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:27.992+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:28.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:28.001 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:35:28.099 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:28.094+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:28.166 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:28.166 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:28.166 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.166 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone17 from 0x1632200 (overlap 0x1632200) to 0x470600 2024-09-19T21:35:28.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:28.199 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:28.192+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:28.300 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:28.298+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:28.307 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.307 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:28.307 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:28.307 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:28.307 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 59, 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:35:27.959118+0000", 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:34:04.640393+0000", 2024-09-19T21:35:28.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5799936, 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:35:28.396 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "luminous", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:35:28.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:28.398 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:28.399 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:28.400 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:28.401 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:28.402 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:35:28.403 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:35:28.404 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:28.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:35:28.406 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.407 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:28.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:28.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:28.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.411 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.25, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.25, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:28.413 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:28.414 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "59", 2024-09-19T21:35:28.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:28.416 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 21, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 59, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:28.417 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:28.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:28.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.419 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:28.419 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:28.419 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:28.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:28.420 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:28.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.1299999952316284, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.1299999952316284, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-19T21:35:28.422 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.423 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 56, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.424 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.429 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:28.430 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:35:28.431 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:35:28.432 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.433 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:28.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.443 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 24, 2024-09-19T21:35:28.444 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.447 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:28.451 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:35:28.452 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.454 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.457 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:28.458 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:28.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:28.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:35:28.469 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [ 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: "pgid": "3.15", 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: "mappings": [ 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: "from": 3, 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: "to": 5 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.478 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:35:27.959118+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:34:04.640393+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 59, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.15', 'mappings': [{'from': 3, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:35:28.478 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-19T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.534266114000005, 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:28.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:28.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:28.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:28.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.488 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15154 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5314560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.824638626, 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056594848000000003, 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15154 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642427+0000", 2024-09-19T21:35:28.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642372+0000", 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999456 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642431+0000", 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8517000000000002e-05 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642439+0000", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1980000000000001e-06 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642445+0000", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3980000000000004e-06 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698816+0000", 2024-09-19T21:35:28.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056370908999999997 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.698937+0000", 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000120976 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699021+0000", 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4206e-05 2024-09-19T21:35:28.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15155 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5328896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.824473298000001, 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056657103, 2024-09-19T21:35:28.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15155 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:28.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642592+0000", 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642450+0000", 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998584 2024-09-19T21:35:28.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642601+0000", 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015027000000000001 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642618+0000", 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7482999999999998e-05 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642623+0000", 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9409999999999998e-06 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699053+0000", 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056430165999999997 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699162+0000", 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000108626 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699249+0000", 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7064999999999994e-05 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15156 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5343232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.8243519, 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056741714999999998, 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15156 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.500 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642713+0000", 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642628+0000", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999146 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642716+0000", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8770000000000006e-05 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642722+0000", 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6860000000000003e-06 2024-09-19T21:35:28.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642726+0000", 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9489999999999998e-06 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699283+0000", 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556510999999997 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699378+0000", 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5079000000000002e-05 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699455+0000", 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7245999999999999e-05 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15157 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5357568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.824070846, 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056672473000000001, 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15157 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.505 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642994+0000", 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642729+0000", 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997349 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.642999+0000", 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026962200000000001 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643008+0000", 2024-09-19T21:35:28.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6589999999999996e-06 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643014+0000", 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.888e-06 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699488+0000", 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056473795 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699590+0000", 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010186100000000001 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699667+0000", 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6627999999999999e-05 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15158 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5371904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:28.510 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.823829109, 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056666994999999998, 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15158 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.511 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643236+0000", 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643020+0000", 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997845 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643241+0000", 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022081999999999999 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643248+0000", 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0389999999999999e-06 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643255+0000", 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7059999999999998e-06 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699701+0000", 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056446488000000003 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699820+0000", 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000119053 2024-09-19T21:35:28.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699903+0000", 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2595000000000005e-05 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15159 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5386240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.823633493999999, 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056686874999999998, 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15159 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643432+0000", 2024-09-19T21:35:28.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643259+0000", 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998279 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643435+0000", 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017585800000000001 2024-09-19T21:35:28.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643442+0000", 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7889999999999997e-06 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643446+0000", 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4710000000000001e-06 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.699931+0000", 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056484178000000003 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700036+0000", 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010582 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700119+0000", 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2052999999999995e-05 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15160 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5400576~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.823531603999999, 2024-09-19T21:35:28.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056812712000000001, 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15160 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643534+0000", 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643450+0000", 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999166 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643536+0000", 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6331999999999999e-05 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643541+0000", 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5900000000000001e-06 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643545+0000", 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9049999999999999e-06 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700149+0000", 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056604067000000001 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700262+0000", 2024-09-19T21:35:28.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000113408 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700346+0000", 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3808999999999997e-05 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15161 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5414912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.527 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.823337102, 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056828916, 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15161 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643728+0000", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643548+0000", 2024-09-19T21:35:28.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998202 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643731+0000", 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018323300000000001 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643737+0000", 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.088e-06 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643742+0000", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.284e-06 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700374+0000", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056632688 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700481+0000", 2024-09-19T21:35:28.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106531 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700557+0000", 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5995999999999995e-05 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15162 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5429248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.532 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.823216845999999, 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056920353, 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15162 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643848+0000", 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643745+0000", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999897 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643852+0000", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106959 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643862+0000", 2024-09-19T21:35:28.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9529999999999994e-06 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643869+0000", 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9199999999999998e-06 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700585+0000", 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056716492 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700685+0000", 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700769+0000", 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3264999999999996e-05 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15163 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5443584~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822708288999999, 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056623993999999997, 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15163 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644357+0000", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.643873+0000", 2024-09-19T21:35:28.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9995165 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644360+0000", 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048695100000000001 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644370+0000", 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6809999999999993e-06 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644374+0000", 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.639e-06 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700825+0000", 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056450137999999997 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700915+0000", 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0080000000000005e-05 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.700981+0000", 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6202000000000001e-05 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15164 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5457920~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822686644000001, 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056756384, 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15164 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644378+0000", 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644380+0000", 2024-09-19T21:35:28.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9640000000000002e-06 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644384+0000", 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4479999999999999e-06 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644388+0000", 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5040000000000004e-06 2024-09-19T21:35:28.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701006+0000", 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056617546999999997 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701079+0000", 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3755000000000001e-05 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701135+0000", 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448999999999997e-05 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15165 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5472256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822672893, 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056848226000000002, 2024-09-19T21:35:28.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15165 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644392+0000", 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:28.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644395+0000", 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9919999999999999e-06 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644401+0000", 2024-09-19T21:35:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2310000000000001e-06 2024-09-19T21:35:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644406+0000", 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1429999999999997e-06 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701150+0000", 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056743891999999997 2024-09-19T21:35:28.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701201+0000", 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0614e-05 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701240+0000", 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9697000000000003e-05 2024-09-19T21:35:28.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15166 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5486592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822548348, 2024-09-19T21:35:28.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056825742999999998, 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15166 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644517+0000", 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644410+0000", 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998937 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644521+0000", 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011017400000000001 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644526+0000", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644530+0000", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701255+0000", 2024-09-19T21:35:28.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056724519000000001 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701292+0000", 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7425000000000002e-05 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701343+0000", 2024-09-19T21:35:28.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0553999999999997e-05 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15167 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5500928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822323774999999, 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056707884, 2024-09-19T21:35:28.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15167 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644741+0000", 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644545+0000", 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998035 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644746+0000", 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200779 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644756+0000", 2024-09-19T21:35:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0916e-05 2024-09-19T21:35:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644761+0000", 2024-09-19T21:35:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.228e-06 2024-09-19T21:35:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701357+0000", 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056595897999999999 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701412+0000", 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5634000000000003e-05 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701449+0000", 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7017999999999997e-05 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15168 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5515264~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.822081664000001, 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056638200999999999, 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15168 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644983+0000", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644775+0000", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997921 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644987+0000", 2024-09-19T21:35:28.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021153200000000001 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.644999+0000", 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2089999999999999e-05 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645004+0000", 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4020000000000004e-06 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701471+0000", 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056467824999999999 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701560+0000", 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8195999999999994e-05 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701622+0000", 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2130000000000003e-05 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15170 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5543936~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.821702316, 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056639429999999998, 2024-09-19T21:35:28.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15170 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645363+0000", 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645362+0000", 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645365+0000", 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0779999999999998e-06 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645368+0000", 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6720000000000002e-06 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645371+0000", 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2480000000000001e-06 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701839+0000", 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056468061 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.701937+0000", 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7979999999999994e-05 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702002+0000", 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4770000000000005e-05 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15171 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5558272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.821426446, 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056556991000000001, 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15171 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645639+0000", 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645374+0000", 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9997354 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645642+0000", 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026817799999999999 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645648+0000", 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4389999999999997e-06 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645653+0000", 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1949999999999996e-06 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702027+0000", 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056374806 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702126+0000", 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8941e-05 2024-09-19T21:35:28.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702196+0000", 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9239999999999994e-05 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15172 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5572608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.584 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.821231816999999, 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056523239000000003, 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15172 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645833+0000", 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645656+0000", 2024-09-19T21:35:28.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998231 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645837+0000", 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018102 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645845+0000", 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0649999999999999e-06 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645850+0000", 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6619999999999996e-06 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702226+0000", 2024-09-19T21:35:28.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056376060999999998 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702303+0000", 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7055e-05 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702357+0000", 2024-09-19T21:35:28.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3517999999999998e-05 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15173 3.d 3:b57eb537:::rbd_data.11066d5522c8.0000000000000004:12 [read 5586944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.821102488999999, 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056512898999999998, 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:28.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15173 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:28.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645963+0000", 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645853+0000", 2024-09-19T21:35:28.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9998908 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645966+0000", 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011243600000000001 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:28.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645972+0000", 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.066e-06 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.645977+0000", 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9320000000000004e-06 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:28.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702371+0000", 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394702999999997 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702428+0000", 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6508000000000002e-05 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.702476+0000", 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7576e-05 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.595 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:28.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:28.596 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.596 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:28.474+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:28.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:28.599 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:28.596+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:28.700 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:28.698+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:28.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:28.716 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.717 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:28.717 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:28.717 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.728 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:28.800 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:28.796+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:28.844 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:28.844 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:28.844 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:28.844 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:28.844 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:28.851 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:28.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:28.869 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:28.886 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:28.902 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:28.896+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:28.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:28.932 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:28.957 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:28.957 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:28.957 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:28.959 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:28.960 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:28.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:29.015 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:29.014+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:29.088 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.088 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.097 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-19T21:35:29.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:29.103 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:29.098+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:29.204 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:29.196+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:29.205 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.205 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:29.235 INFO:teuthology.orchestra.run.smithi122.stdout:2835 read 0x117bc00 thru 0x11819a8 (0x5da9 bytes) 2024-09-19T21:35:29.237 INFO:teuthology.orchestra.run.smithi122.stdout:2836 punch from 0x8bce00 to 0x8cac00, (0xde00 bytes) 2024-09-19T21:35:29.304 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:29.300+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:29.346 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.346 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:29.405 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:29.400+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:29.447 INFO:teuthology.orchestra.run.smithi122.stdout:2837 write 0xbf55800 thru 0xbf59784 (0x3f85 bytes) 2024-09-19T21:35:29.467 INFO:teuthology.orchestra.run.smithi122.stdout:2838 trunc from 0xbf59800 to 0x2dba200 2024-09-19T21:35:29.506 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:29.500+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:29.511 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.511 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.511 INFO:teuthology.orchestra.run.smithi122.stdout:2841 read 0x101c200 thru 0x102ab53 (0xe954 bytes) 2024-09-19T21:35:29.516 INFO:teuthology.orchestra.run.smithi122.stdout:2843 read 0x2c46c00 thru 0x2c4f5d3 (0x89d4 bytes) 2024-09-19T21:35:29.516 INFO:teuthology.orchestra.run.smithi122.stdout:2844 read 0x24b3600 thru 0x24ba8c1 (0x72c2 bytes) 2024-09-19T21:35:29.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:29.527 INFO:teuthology.orchestra.run.smithi122.stdout:2845 trunc from 0x2dba200 to 0x7309200 2024-09-19T21:35:29.555 INFO:teuthology.orchestra.run.smithi122.stdout:2850 trunc from 0x7309200 to 0x3159200 2024-09-19T21:35:29.575 INFO:teuthology.orchestra.run.smithi122.stdout:2851 read 0x1c4a400 thru 0x1c4e796 (0x4397 bytes) 2024-09-19T21:35:29.577 INFO:teuthology.orchestra.run.smithi122.stdout:2854 read 0x2eaf800 thru 0x2eb769f (0x7ea0 bytes) 2024-09-19T21:35:29.581 INFO:teuthology.orchestra.run.smithi122.stdout:2857 trunc from 0x3159200 to 0x186f800 2024-09-19T21:35:29.603 INFO:teuthology.orchestra.run.smithi122.stdout:2858 punch from 0xdd1400 to 0xdd7e00, (0x6a00 bytes) 2024-09-19T21:35:29.605 INFO:teuthology.orchestra.run.smithi122.stdout:2859 punch from 0x943c00 to 0x946600, (0x2a00 bytes) 2024-09-19T21:35:29.605 INFO:teuthology.orchestra.run.smithi122.stdout:2860 punch from 0x1100c00 to 0x110f800, (0xec00 bytes) 2024-09-19T21:35:29.606 INFO:teuthology.orchestra.run.smithi122.stdout:2864 punch from 0x16dea00 to 0x16e5e00, (0x7400 bytes) 2024-09-19T21:35:29.606 INFO:teuthology.orchestra.run.smithi122.stdout:2865 write 0x6bd800 thru 0x6cc471 (0xec72 bytes) 2024-09-19T21:35:29.607 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:29.600+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:29.643 INFO:teuthology.orchestra.run.smithi122.stdout:2866 write 0x4dda600 thru 0x4de08ee (0x62ef bytes) 2024-09-19T21:35:29.645 INFO:teuthology.orchestra.run.smithi122.stdout:2867 write 0x1ea8a00 thru 0x1eaf8c9 (0x6eca bytes) 2024-09-19T21:35:29.648 INFO:teuthology.orchestra.run.smithi122.stdout:2869 clone 19 order 18 su 8192 sc 10 2024-09-19T21:35:29.681 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.681 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:29.737 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:29.732+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:29.801 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.801 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.808 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:29.806+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:29.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:29.910 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:29.906+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:29.919 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:29.919 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:29.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:30.002 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:35:30.010 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:30.004+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:30.040 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.040 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:30.111 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:30.110+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:30.162 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.162 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:30.188 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone18 from 0x4de0a00 (overlap 0x1632200) to 0x13f9e00 2024-09-19T21:35:30.212 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:30.204+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:30.283 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.283 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:30.313 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:30.308+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:30.400 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.400 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:30.456 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:30.454+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:30.513 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.513 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.516 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:30.514+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:30.523 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:30.617 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:30.612+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:30.633 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.633 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.643 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:30.718 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:30.712+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:30.757 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.757 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:30.818 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:30.814+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:30.884 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:30.884 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:30.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:30.935 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:30.934+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:30.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:30.997 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:35:31.052 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:31.050+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:31.055 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:31.056 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:31.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:31.115 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:31.121 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:31.116+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:31.142 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:31.163 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:31.168 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:31.168 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:31.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:31.185 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:31.208 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:31.222 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:31.218+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:31.231 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:31.233 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:31.234 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:31.284 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.052254626, 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:31.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578226817999997, 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578196990999999, 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:31.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:31.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578187305, 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.303 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578178979, 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:31.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:31.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.308 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578168587, 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.578158090999999, 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:31.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.575714003000002, 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:31.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:31.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.550496617, 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:31.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:31.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:31.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:31.329 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15066 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6164480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654901926000001, 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029164998000000001, 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15066 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624793+0000", 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624797+0000", 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4580000000000002e-06 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624803+0000", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6800000000000004e-06 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624809+0000", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8969999999999996e-06 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653805+0000", 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028996141999999999 2024-09-19T21:35:31.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653889+0000", 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3906000000000006e-05 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653959+0000", 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9604000000000006e-05 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.334 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15067 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6178816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654873317, 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029307910999999999, 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15067 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624831+0000", 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2019999999999993e-06 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624840+0000", 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5639999999999997e-06 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624846+0000", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4740000000000002e-06 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653998+0000", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029151795000000001 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654062+0000", 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4289999999999996e-05 2024-09-19T21:35:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654130+0000", 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8034000000000006e-05 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15068 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6193152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654845315999999, 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029394144000000001, 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15068 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624853+0000", 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8339999999999999e-06 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624860+0000", 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7689999999999999e-06 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624866+0000", 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0639999999999999e-06 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654143+0000", 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277609999999999 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654189+0000", 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5892e-05 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654245+0000", 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5383000000000002e-05 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.344 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15069 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.65482497, 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029542570000000001, 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15069 2024-09-19T21:35:31.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624870+0000", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624873+0000", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624880+0000", 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:31.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624886+0000", 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654272+0000", 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029385674000000001 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654328+0000", 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6156999999999998e-05 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654413+0000", 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999998e-05 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15070 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654804781999999, 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02965835, 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15070 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624894+0000", 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9900000000000002e-06 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624903+0000", 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1220000000000001e-06 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624917+0000", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3973e-05 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654426+0000", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029508869 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654475+0000", 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9274999999999998e-05 2024-09-19T21:35:31.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654549+0000", 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4492000000000004e-05 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15071 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6236160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:31.354 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654772748999999, 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029922358, 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15071 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:31.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624922+0000", 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624926+0000", 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2650000000000001e-06 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624933+0000", 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624939+0000", 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9819999999999996e-06 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654596+0000", 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029657246000000002 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654708+0000", 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112194 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654845+0000", 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013685999999999999 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15072 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6250496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654751337, 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030057148999999998, 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.359 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15072 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:31.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624947+0000", 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.326e-06 2024-09-19T21:35:31.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624954+0000", 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6590000000000001e-06 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624959+0000", 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3630000000000002e-06 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654863+0000", 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029903855999999999 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654917+0000", 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4487000000000002e-05 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655002+0000", 2024-09-19T21:35:31.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4300000000000003e-05 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15073 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6264832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654730334, 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030212118999999999, 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15073 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624964+0000", 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624968+0000", 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.16e-06 2024-09-19T21:35:31.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624975+0000", 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9600000000000003e-06 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624981+0000", 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7980000000000002e-06 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655037+0000", 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030056058 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655093+0000", 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6223e-05 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655177+0000", 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4654000000000006e-05 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.368 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15074 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.654710109, 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030446608, 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15074 2024-09-19T21:35:31.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624985+0000", 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624988+0000", 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1370000000000002e-06 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625010+0000", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1778000000000001e-05 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625017+0000", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5259999999999999e-06 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655204+0000", 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030186983000000001 2024-09-19T21:35:31.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655303+0000", 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655432+0000", 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012875899999999999 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18639 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 731136~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.257732638, 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029303412000000001, 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18639 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021964+0000", 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021967+0000", 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021970+0000", 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2349999999999999e-06 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051070+0000", 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029100002 2024-09-19T21:35:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051159+0000", 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9033e-05 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051266+0000", 2024-09-19T21:35:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107069 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18640 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 745472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.257712267999999, 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029524404000000001, 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18640 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021985+0000", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9750000000000001e-06 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021988+0000", 2024-09-19T21:35:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9720000000000001e-06 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021991+0000", 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6369999999999999e-06 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051317+0000", 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029326609 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051398+0000", 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0705999999999996e-05 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051508+0000", 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109948 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:31.384 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:31.385 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-19T21:35:31.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:31.386 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:31.382+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:31.486 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:31.482+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:31.491 INFO:teuthology.orchestra.run.smithi122.stdout:2871 read 0x2ca1600 thru 0x2ca410e (0x2b0f bytes) 2024-09-19T21:35:31.493 INFO:teuthology.orchestra.run.smithi122.stdout:2873 read 0x35fb800 thru 0x35fefaf (0x37b0 bytes) 2024-09-19T21:35:31.494 INFO:teuthology.orchestra.run.smithi122.stdout:2874 write 0x158fa00 thru 0x1593ac4 (0x40c5 bytes) 2024-09-19T21:35:31.495 INFO:teuthology.orchestra.run.smithi122.stdout:2875 write 0x1b1e000 thru 0x1b20f10 (0x2f11 bytes) 2024-09-19T21:35:31.497 INFO:teuthology.orchestra.run.smithi122.stdout:2876 write 0x28cc400 thru 0x28d992d (0xd52e bytes) 2024-09-19T21:35:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:31.498 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:31.498 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:31.498 INFO:teuthology.orchestra.run.smithi122.stdout:2878 punch from 0x16d0a00 to 0x16dcc00, (0xc200 bytes) 2024-09-19T21:35:31.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:31.587 INFO:teuthology.orchestra.run.smithi122.stdout:2879 read 0x49b2a00 thru 0x49b2cd4 (0x2d5 bytes) 2024-09-19T21:35:31.587 INFO:teuthology.orchestra.run.smithi122.stdout:2880 read 0x4683200 thru 0x4692668 (0xf469 bytes) 2024-09-19T21:35:31.587 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:31.586+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:31.606 INFO:teuthology.orchestra.run.smithi122.stdout:2881 trunc from 0x4de0a00 to 0x2e97a00 2024-09-19T21:35:31.613 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:31.613 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:31.613 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:31.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:31.689 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:31.684+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:31.715 INFO:teuthology.orchestra.run.smithi122.stdout:2882 write 0xabf2c00 thru 0xabf9502 (0x6903 bytes) 2024-09-19T21:35:31.717 INFO:teuthology.orchestra.run.smithi122.stdout:2886 read 0x7a8e00 thru 0x7ac838 (0x3a39 bytes) 2024-09-19T21:35:31.721 INFO:teuthology.orchestra.run.smithi122.stdout:2888 read 0x98c4600 thru 0x98cb18c (0x6b8d bytes) 2024-09-19T21:35:31.721 INFO:teuthology.orchestra.run.smithi122.stdout:2889 punch from 0x692fc00 to 0x6938e00, (0x9200 bytes) 2024-09-19T21:35:31.722 INFO:teuthology.orchestra.run.smithi122.stdout:2890 read 0x1d6d000 thru 0x1d73137 (0x6138 bytes) 2024-09-19T21:35:31.725 INFO:teuthology.orchestra.run.smithi122.stdout:2892 punch from 0x39b1000 to 0x39b7000, (0x6000 bytes) 2024-09-19T21:35:31.725 INFO:teuthology.orchestra.run.smithi122.stdout:2893 write 0x8a1c000 thru 0x8a2bba2 (0xfba3 bytes) 2024-09-19T21:35:31.726 INFO:teuthology.orchestra.run.smithi122.stdout:2898 read 0xfc800 thru 0x107bc3 (0xb3c4 bytes) 2024-09-19T21:35:31.731 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:31.731 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:31.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473788760000001, 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.736 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473775221, 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473749480999999, 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473735266999999, 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:31.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.751 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473714350000002, 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.752 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:31.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:31.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:31.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:31.758 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473698974000001, 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:31.764 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.765 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473684045999999, 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473631426000001, 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473581847999998, 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:31.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473564632999999, 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:31.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:31.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.787 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473545495, 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:31.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473530647, 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:31.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.797 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.473507653999999, 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:31.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:31.802 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.472775362, 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:31.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:31.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.472725052000001, 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:31.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:31.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:31.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.472662740000001, 2024-09-19T21:35:31.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:31.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:31.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.471332106999999, 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.471196529, 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:31.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:31.826 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.471119770000001, 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.471061045999999, 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:31.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:31.836 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:31.837 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:31.838 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:31.834+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:31.839 INFO:teuthology.orchestra.run.smithi122.stdout:2899 read 0xa1dfe00 thru 0xa1ebc2d (0xbe2e bytes) 2024-09-19T21:35:31.839 INFO:teuthology.orchestra.run.smithi122.stdout:2900 read 0x5016000 thru 0x5023ebe (0xdebf bytes) 2024-09-19T21:35:31.839 INFO:teuthology.orchestra.run.smithi122.stdout:2902 read 0x541c800 thru 0x542ba74 (0xf275 bytes) 2024-09-19T21:35:31.839 INFO:teuthology.orchestra.run.smithi122.stdout:2905 write 0xf29a00 thru 0xf3673c (0xcd3d bytes) 2024-09-19T21:35:31.840 INFO:teuthology.orchestra.run.smithi122.stdout:2906 read 0x8834e00 thru 0x8844157 (0xf358 bytes) 2024-09-19T21:35:31.840 INFO:teuthology.orchestra.run.smithi122.stdout:2908 clone 20 order 25 su 4194304 sc 10 2024-09-19T21:35:31.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:31.938 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:31.932+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:31.997 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:31.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:31.997 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:31.997 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:32.061 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:32.056+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:32.122 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:32.122 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:32.139 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:32.132+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:32.240 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone19 2024-09-19T21:35:32.241 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:32.236+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4082 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0420980249999996, 2024-09-19T21:35:32.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068451995000000002, 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4082 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196962+0000", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.734e-06 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196965+0000", 2024-09-19T21:35:32.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6979999999999998e-06 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196972+0000", 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9369999999999998e-06 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.264911+0000", 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067938800999999993 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.264995+0000", 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4637999999999998e-05 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265413+0000", 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041739199999999999 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4083 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0420845720000003, 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068703602000000003, 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4083 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:32.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196976+0000", 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8470000000000001e-06 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196979+0000", 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.038e-06 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196985+0000", 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265450+0000", 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068464982999999993 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265554+0000", 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104085 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265678+0000", 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000123502 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4095 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0415356859999996, 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068425451999999998, 2024-09-19T21:35:32.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4095 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197526+0000", 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3139999999999999e-06 2024-09-19T21:35:32.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197532+0000", 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4250000000000003e-06 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197535+0000", 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1300000000000001e-06 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265709+0000", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068173553999999997 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265800+0000", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1246e-05 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265949+0000", 2024-09-19T21:35:32.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000148317 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4097 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0415204310000004, 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068888300999999999, 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4097 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197540+0000", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.283e-06 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197544+0000", 2024-09-19T21:35:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.083e-06 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197547+0000", 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.005e-06 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265987+0000", 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068440300999999995 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266046+0000", 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8983000000000001e-05 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266427+0000", 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038087700000000001 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4099 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0415097920000003, 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069491868999999998, 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4099 2024-09-19T21:35:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197567+0000", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8372999999999998e-05 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197576+0000", 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2989999999999997e-06 2024-09-19T21:35:32.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197581+0000", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8050000000000003e-06 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266471+0000", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068890122999999998 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266606+0000", 2024-09-19T21:35:32.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013495799999999999 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267041+0000", 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043451999999999998 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4101 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.267 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0414718260000004, 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069876111000000005, 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4101 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197586+0000", 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197589+0000", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0790000000000001e-06 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197599+0000", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6039999999999999e-06 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197605+0000", 2024-09-19T21:35:32.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267096+0000", 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069491154999999999 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267162+0000", 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6161000000000006e-05 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267463+0000", 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000301238 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4109 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0414497029999996, 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070017945999999998, 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4109 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:32.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197611+0000", 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.712e-06 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197621+0000", 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4250000000000004e-06 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197627+0000", 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7259999999999999e-06 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267488+0000", 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069861147999999998 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267547+0000", 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9497000000000003e-05 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267627+0000", 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9831000000000002e-05 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.041365699, 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:35:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:35:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.040785788, 2024-09-19T21:35:32.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:35:32.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:35:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.040723141, 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:35:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0406568209999998, 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:35:32.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.039953143, 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:35:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0398803210000001, 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:35:32.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0398665830000002, 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:35:32.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:35:32.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0398488940000004, 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.313 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:35:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0397985079999996, 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:35:32.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:35:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0397716189999997, 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:35:32.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.327 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0397310429999997, 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0397094290000002, 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:35:32.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.337 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0396902299999997, 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:35:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.342 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.343 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:32.342+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:32.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:32.444 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:32.442+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:32.521 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:32.521 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:32.522 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.533 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:32.546 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:32.542+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:32.645 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:32.645 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.646 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:32.640+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:32.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:32.747 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:32.746+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:32.763 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:32.763 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:32.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:32.763 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:32.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.010651838000001, 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.764 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:32.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:32.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38478 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3237888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.173950415, 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050662393999999999, 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38478 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585323+0000", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585326+0000", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.36e-06 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585339+0000", 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2836999999999999e-05 2024-09-19T21:35:32.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585349+0000", 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0227999999999999e-05 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635814+0000", 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050465015000000002 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635883+0000", 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8490000000000003e-05 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.635986+0000", 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000102972 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38479 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.775 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.173741335000001, 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050617212000000002, 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38479 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585532+0000", 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585354+0000", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998221 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585538+0000", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000184007 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585548+0000", 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1190000000000003e-06 2024-09-19T21:35:32.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.585555+0000", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1380000000000001e-06 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636018+0000", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050463348999999998 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636091+0000", 2024-09-19T21:35:32.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3081e-05 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.636150+0000", 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8387000000000002e-05 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38745 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3299328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.156987579000001, 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050507966000000001, 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38745 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602286+0000", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602096+0000", 2024-09-19T21:35:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998102 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602292+0000", 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019607700000000001 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602303+0000", 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0248000000000001e-05 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602320+0000", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7898e-05 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652572+0000", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050251704000000001 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652684+0000", 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00011210299999999999 2024-09-19T21:35:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652794+0000", 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000109832 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38746 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3313664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.156812456000001, 2024-09-19T21:35:32.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050552997000000002, 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38746 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602461+0000", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602337+0000", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999876 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602467+0000", 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000129827 2024-09-19T21:35:32.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602478+0000", 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.057e-05 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602503+0000", 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5826e-05 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652823+0000", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050319567000000003 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.652916+0000", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2750999999999993e-05 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653014+0000", 2024-09-19T21:35:32.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8435000000000002e-05 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38747 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3328000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.156713058999999, 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050666628999999998, 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38747 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602561+0000", 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602511+0000", 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999504 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602566+0000", 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4653e-05 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602574+0000", 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1799999999999996e-06 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602581+0000", 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0960000000000003e-06 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653042+0000", 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050461113000000002 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653135+0000", 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2662000000000002e-05 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653227+0000", 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.2315e-05 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.794 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38748 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.156525203999999, 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050685503999999999, 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38748 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.795 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602748+0000", 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602587+0000", 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998384 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602755+0000", 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000167875 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602781+0000", 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6815e-05 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602800+0000", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8644999999999999e-05 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653253+0000", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050452664000000001 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653338+0000", 2024-09-19T21:35:32.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5718999999999997e-05 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653434+0000", 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5524000000000002e-05 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38749 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3356672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.156252731, 2024-09-19T21:35:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050628085000000003, 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38749 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603021+0000", 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.602862+0000", 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998407 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603026+0000", 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016469600000000001 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603037+0000", 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0208e-05 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603049+0000", 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2133999999999999e-05 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653466+0000", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050417464000000002 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653553+0000", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6384999999999994e-05 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653649+0000", 2024-09-19T21:35:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6376e-05 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38750 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3371008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.155965806999999, 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050551984000000001, 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.804 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38750 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603308+0000", 2024-09-19T21:35:32.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603056+0000", 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997478 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603313+0000", 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025696099999999999 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603333+0000", 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0117000000000001e-05 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603341+0000", 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653674+0000", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050333152999999999 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653765+0000", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1237999999999996e-05 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653860+0000", 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4347999999999999e-05 2024-09-19T21:35:32.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38751 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3385344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.155744201999999, 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050533023000000003, 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38751 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603529+0000", 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603350+0000", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998207 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603534+0000", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00018449199999999999 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603544+0000", 2024-09-19T21:35:32.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7489999999999993e-06 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.603552+0000", 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8269999999999998e-06 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653888+0000", 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050335574000000001 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.653975+0000", 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7108999999999996e-05 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654062+0000", 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7756000000000005e-05 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38754 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3428352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.155163418000001, 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050586417000000002, 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38754 2024-09-19T21:35:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604110+0000", 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604109+0000", 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604114+0000", 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4379999999999997e-06 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604121+0000", 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7170000000000008e-06 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604127+0000", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9889999999999997e-06 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654520+0000", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050392827000000001 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654605+0000", 2024-09-19T21:35:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5191999999999993e-05 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654697+0000", 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1293999999999997e-05 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38755 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3442688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.155009034000001, 2024-09-19T21:35:32.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050639968, 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38755 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:32.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604265+0000", 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604132+0000", 2024-09-19T21:35:32.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9998674 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604276+0000", 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014431399999999999 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604295+0000", 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8457999999999998e-05 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.604303+0000", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9820000000000008e-06 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654725+0000", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050422012000000002 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654811+0000", 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5921000000000006e-05 2024-09-19T21:35:32.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.654905+0000", 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3906000000000005e-05 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38775 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3729408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.150842558000001, 2024-09-19T21:35:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050773897999999998, 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38775 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608431+0000", 2024-09-19T21:35:32.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608432+0000", 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.424e-06 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608435+0000", 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.608438+0000", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.774e-06 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.658966+0000", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050528196999999997 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659086+0000", 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000119994 2024-09-19T21:35:32.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.659205+0000", 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000118849 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.828 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38789 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3930112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.147097486, 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050834478000000002, 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38789 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612176+0000", 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612175+0000", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612178+0000", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9519999999999999e-06 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612185+0000", 2024-09-19T21:35:32.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2740000000000004e-06 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612192+0000", 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8210000000000004e-06 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662685+0000", 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050493581000000003 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.662845+0000", 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016009299999999999 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663011+0000", 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016543800000000001 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38790 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3944448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.147065952, 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051021812, 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38790 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612208+0000", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612207+0000", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612210+0000", 2024-09-19T21:35:32.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0400000000000001e-06 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612217+0000", 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3159999999999999e-06 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612223+0000", 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663043+0000", 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050820585000000001 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663124+0000", 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0409999999999998e-05 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663230+0000", 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105659 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38791 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 3958784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.147046121000001, 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051185702, 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38791 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612228+0000", 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612227+0000", 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612238+0000", 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1326e-05 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612244+0000", 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1430000000000003e-06 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.612250+0000", 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5249999999999996e-06 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663257+0000", 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051007515000000003 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663335+0000", 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7265e-05 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.663413+0000", 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.852e-05 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.845 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38804 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4145152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.143303825, 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050530841, 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38804 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615970+0000", 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615969+0000", 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615973+0000", 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.5209999999999998e-06 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615981+0000", 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5269999999999998e-06 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615987+0000", 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5600000000000001e-06 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666333+0000", 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050346683000000003 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666413+0000", 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9437000000000005e-05 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666501+0000", 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7766e-05 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38805 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.143282315, 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050694516000000002, 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38805 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:32.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615991+0000", 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.615994+0000", 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.762e-06 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616006+0000", 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2201999999999999e-05 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616015+0000", 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4730000000000006e-06 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666526+0000", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050510477999999998 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666598+0000", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1769999999999999e-05 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666686+0000", 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8330000000000003e-05 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38806 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4173824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.143253325, 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0508669, 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38806 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616020+0000", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616023+0000", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2260000000000001e-06 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616031+0000", 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8760000000000005e-06 2024-09-19T21:35:32.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616036+0000", 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.848e-06 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666710+0000", 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050674286999999998 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666789+0000", 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8916000000000001e-05 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666887+0000", 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8210999999999999e-05 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:38807 3.17 3:ece43bee:::rbd_data.11066d5522c8.0000000000000006:12 [read 4188160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 15.14323351, 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.051052267999999998, 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 38807 2024-09-19T21:35:32.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616040+0000", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616042+0000", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2620000000000001e-06 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616048+0000", 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0310000000000004e-06 2024-09-19T21:35:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.616059+0000", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0855e-05 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.666927+0000", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.050868501000000003 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667017+0000", 2024-09-19T21:35:32.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9625999999999999e-05 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:17.667092+0000", 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5337000000000001e-05 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:32.864 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:32.866 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:32.860+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:32.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:32.966 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:32.962+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:32.983 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:32.983 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:32.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:33.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:35:33.068 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:33.066+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:33.096 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:33.096 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:33.096 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:33.096 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:33.096 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:33.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:33.168 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:33.164+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:33.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.89787780699999997, 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:33.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.89786549900000001, 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:33.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:33.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:33.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86513426000000004, 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:33.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.635376567, 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.276 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:33.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:33.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63535313199999999, 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:33.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:33.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:33.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63533140399999999, 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:33.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63531067200000002, 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:33.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.294 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63527623600000005, 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63523343899999996, 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:33.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.63000792900000002, 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62998563100000005, 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.309 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:33.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62995477600000005, 2024-09-19T21:35:33.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:33.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:33.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62991206700000002, 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:33.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62988807999999996, 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:33.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.326 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62985667599999995, 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:33.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62983151000000004, 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.331 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:33.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62978643199999995, 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:33.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:33.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62975968800000004, 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.340 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.343 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62974639600000004, 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.344 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.62972947700000004, 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:33.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:33.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:33.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:33.352 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:33.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:33.354 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:33.348+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:33.384 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:33.421 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:33.455 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:33.450+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:33.457 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:33.479 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:33.480 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:33.480 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-19T21:35:33.480 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-19T21:35:33.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-19T21:35:33.491 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:33.494 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:33.494 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:33.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:33.518 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:33.543 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:33.545 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:33.555 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:33.554+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:33.632 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:33.632 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:33.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:33.656 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:33.652+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:33.731 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-19T21:35:33.754 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:33.754 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:33.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:33.752+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:33.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:33.858 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:33.852+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:33.876 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:33.876 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:33.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:33.943 INFO:teuthology.orchestra.run.smithi122.stdout:2909 write 0x7ecb200 thru 0x7ed6867 (0xb668 bytes) 2024-09-19T21:35:33.944 INFO:teuthology.orchestra.run.smithi122.stdout:2916 punch from 0x89e9600 to 0x89e9e00, (0x800 bytes) 2024-09-19T21:35:33.959 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:33.954+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:33.993 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:33.994 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:34.013 INFO:teuthology.orchestra.run.smithi122.stdout:2920 trunc from 0xabf9600 to 0x9141600 2024-09-19T21:35:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-19T21:35:34.023 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:35:34.059 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:34.058+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:34.113 INFO:teuthology.orchestra.run.smithi122.stdout:2921 write 0xbef6600 thru 0xbefa7e0 (0x41e1 bytes) 2024-09-19T21:35:34.121 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.121 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.131 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:34.137 INFO:teuthology.orchestra.run.smithi122.stdout:2923 trunc from 0xbefa800 to 0x782a200 2024-09-19T21:35:34.180 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:34.176+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:34.202 INFO:teuthology.orchestra.run.smithi122.stdout:2924 write 0x4a3f600 thru 0x4a43427 (0x3e28 bytes) 2024-09-19T21:35:34.205 INFO:teuthology.orchestra.run.smithi122.stdout:2928 write 0x4ee0400 thru 0x4ee14e3 (0x10e4 bytes) 2024-09-19T21:35:34.206 INFO:teuthology.orchestra.run.smithi122.stdout:2929 write 0x4ba0c00 thru 0x4badb61 (0xcf62 bytes) 2024-09-19T21:35:34.207 INFO:teuthology.orchestra.run.smithi122.stdout:2930 read 0x63d0000 thru 0x63d501e (0x501f bytes) 2024-09-19T21:35:34.247 INFO:teuthology.orchestra.run.smithi122.stdout:2931 trunc from 0x782a200 to 0x204e000 2024-09-19T21:35:34.255 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.255 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.261 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:34.256+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:34.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:34.292 INFO:teuthology.orchestra.run.smithi122.stdout:2932 punch from 0x107fc00 to 0x1086c00, (0x7000 bytes) 2024-09-19T21:35:34.325 INFO:teuthology.orchestra.run.smithi122.stdout:2934 write 0x61dac00 thru 0x61e7d2b (0xd12c bytes) 2024-09-19T21:35:34.356 INFO:teuthology.orchestra.run.smithi122.stdout:2937 write 0x8095400 thru 0x809be5e (0x6a5f bytes) 2024-09-19T21:35:34.359 INFO:teuthology.orchestra.run.smithi122.stdout:2941 punch from 0x42b8600 to 0x42c2400, (0x9e00 bytes) 2024-09-19T21:35:34.359 INFO:teuthology.orchestra.run.smithi122.stdout:2942 punch from 0x3dae200 to 0x3dae400, (0x200 bytes) 2024-09-19T21:35:34.360 INFO:teuthology.orchestra.run.smithi122.stdout:2943 write 0x1d5dc00 thru 0x1d6cf1e (0xf31f bytes) 2024-09-19T21:35:34.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:34.356+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:34.374 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.374 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:34.395 INFO:teuthology.orchestra.run.smithi122.stdout:2944 write 0xbafec00 thru 0xbb04d02 (0x6103 bytes) 2024-09-19T21:35:34.413 INFO:teuthology.orchestra.run.smithi122.stdout:2946 trunc from 0xbb04e00 to 0xdb29600 2024-09-19T21:35:34.439 INFO:teuthology.orchestra.run.smithi122.stdout:2948 write 0x7ba1e00 thru 0x7baee38 (0xd039 bytes) 2024-09-19T21:35:34.462 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:34.456+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:34.476 INFO:teuthology.orchestra.run.smithi122.stdout:2951 trunc from 0xdb29600 to 0x125c600 2024-09-19T21:35:34.543 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.543 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:34.556 INFO:teuthology.orchestra.run.smithi122.stdout:2952 read 0x12800 thru 0x1bd8b (0x958c bytes) 2024-09-19T21:35:34.563 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:34.558+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:34.563 INFO:teuthology.orchestra.run.smithi122.stdout:2953 read 0x123b000 thru 0x12492db (0xe2dc bytes) 2024-09-19T21:35:34.564 INFO:teuthology.orchestra.run.smithi122.stdout:2954 read 0x11c0200 thru 0x11c6c45 (0x6a46 bytes) 2024-09-19T21:35:34.602 INFO:teuthology.orchestra.run.smithi122.stdout:2955 write 0x74c3400 thru 0x74d19d0 (0xe5d1 bytes) 2024-09-19T21:35:34.637 INFO:teuthology.orchestra.run.smithi122.stdout:2958 write 0xaa7ba00 thru 0xaa7d827 (0x1e28 bytes) 2024-09-19T21:35:34.641 INFO:teuthology.orchestra.run.smithi122.stdout:2960 read 0xa129400 thru 0xa12b120 (0x1d21 bytes) 2024-09-19T21:35:34.644 INFO:teuthology.orchestra.run.smithi122.stdout:2962 punch from 0x86b800 to 0x86d600, (0x1e00 bytes) 2024-09-19T21:35:34.644 INFO:teuthology.orchestra.run.smithi122.stdout:2963 read 0x4802a00 thru 0x480e140 (0xb741 bytes) 2024-09-19T21:35:34.645 INFO:teuthology.orchestra.run.smithi122.stdout:2965 write 0x7be5a00 thru 0x7bf41bc (0xe7bd bytes) 2024-09-19T21:35:34.647 INFO:teuthology.orchestra.run.smithi122.stdout:2966 write 0x812de00 thru 0x81313df (0x35e0 bytes) 2024-09-19T21:35:34.651 INFO:teuthology.orchestra.run.smithi122.stdout:2968 read 0x6a21400 thru 0x6a27445 (0x6046 bytes) 2024-09-19T21:35:34.651 INFO:teuthology.orchestra.run.smithi122.stdout:2969 read 0x43d800 thru 0x44d73d (0xff3e bytes) 2024-09-19T21:35:34.653 INFO:teuthology.orchestra.run.smithi122.stdout:2971 write 0x7a68c00 thru 0x7a69ca3 (0x10a4 bytes) 2024-09-19T21:35:34.654 INFO:teuthology.orchestra.run.smithi122.stdout:2972 punch from 0x1097000 to 0x109da00, (0x6a00 bytes) 2024-09-19T21:35:34.655 INFO:teuthology.orchestra.run.smithi122.stdout:2974 read 0x95f3600 thru 0x9602a7a (0xf47b bytes) 2024-09-19T21:35:34.655 INFO:teuthology.orchestra.run.smithi122.stdout:2976 read 0x1593200 thru 0x159da62 (0xa863 bytes) 2024-09-19T21:35:34.655 INFO:teuthology.orchestra.run.smithi122.stdout:2977 read 0x31f8200 thru 0x32018d4 (0x96d5 bytes) 2024-09-19T21:35:34.659 INFO:teuthology.orchestra.run.smithi122.stdout:2983 read 0x8606800 thru 0x860f5b4 (0x8db5 bytes) 2024-09-19T21:35:34.660 INFO:teuthology.orchestra.run.smithi122.stdout:2984 punch from 0xa0cc00 to 0xa1c800, (0xfc00 bytes) 2024-09-19T21:35:34.663 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.664 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.664 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:34.662+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:34.671 INFO:teuthology.orchestra.run.smithi122.stdout:2986 trunc from 0xaa7da00 to 0x536e600 2024-09-19T21:35:34.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:34.699 INFO:teuthology.orchestra.run.smithi122.stdout:2989 write 0x499e400 thru 0x49a21fd (0x3dfe bytes) 2024-09-19T21:35:34.702 INFO:teuthology.orchestra.run.smithi122.stdout:2994 write 0x2db5a00 thru 0x2dc03e2 (0xa9e3 bytes) 2024-09-19T21:35:34.703 INFO:teuthology.orchestra.run.smithi122.stdout:2999 read 0x386fc00 thru 0x387b02a (0xb42b bytes) 2024-09-19T21:35:34.711 INFO:teuthology.orchestra.run.smithi122.stdout:3000 trunc from 0x536e600 to 0x24bd400 2024-09-19T21:35:34.744 INFO:teuthology.orchestra.run.smithi122.stdout:3001 read 0xb9da00 thru 0xba702b (0x962c bytes) 2024-09-19T21:35:34.764 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:34.762+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:34.778 INFO:teuthology.orchestra.run.smithi122.stdout:3002 write 0x485a400 thru 0x4864912 (0xa513 bytes) 2024-09-19T21:35:34.780 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.780 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.780 INFO:teuthology.orchestra.run.smithi122.stdout:3003 write 0xda5c00 thru 0xdb4c5c (0xf05d bytes) 2024-09-19T21:35:34.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:34.821 INFO:teuthology.orchestra.run.smithi122.stdout:3004 write 0xb0d9000 thru 0xb0e7e28 (0xee29 bytes) 2024-09-19T21:35:34.824 INFO:teuthology.orchestra.run.smithi122.stdout:3005 write 0x83a3e00 thru 0x83b2d7f (0xef80 bytes) 2024-09-19T21:35:34.825 INFO:teuthology.orchestra.run.smithi122.stdout:3009 write 0x4485600 thru 0x448f2c2 (0x9cc3 bytes) 2024-09-19T21:35:34.825 INFO:teuthology.orchestra.run.smithi122.stdout:3011 punch from 0x747a00 to 0x74ac00, (0x3200 bytes) 2024-09-19T21:35:34.827 INFO:teuthology.orchestra.run.smithi122.stdout:3013 punch from 0x56dc000 to 0x56e9400, (0xd400 bytes) 2024-09-19T21:35:34.827 INFO:teuthology.orchestra.run.smithi122.stdout:3014 read 0x8382600 thru 0x8384853 (0x2254 bytes) 2024-09-19T21:35:34.831 INFO:teuthology.orchestra.run.smithi122.stdout:3015 write 0x8daec00 thru 0x8db2bc6 (0x3fc7 bytes) 2024-09-19T21:35:34.831 INFO:teuthology.orchestra.run.smithi122.stdout:3017 punch from 0x1528200 to 0x1535600, (0xd400 bytes) 2024-09-19T21:35:34.832 INFO:teuthology.orchestra.run.smithi122.stdout:3018 trunc from 0xb0e8000 to 0xb342200 2024-09-19T21:35:34.861 INFO:teuthology.orchestra.run.smithi122.stdout:3020 write 0x2526a00 thru 0x2530c1b (0xa21c bytes) 2024-09-19T21:35:34.863 INFO:teuthology.orchestra.run.smithi122.stdout:3021 write 0x32da200 thru 0x32e1cca (0x7acb bytes) 2024-09-19T21:35:34.864 INFO:teuthology.orchestra.run.smithi122.stdout:3022 read 0xdd600 thru 0xe19c5 (0x43c6 bytes) 2024-09-19T21:35:34.864 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:34.856+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:34.870 INFO:teuthology.orchestra.run.smithi122.stdout:3023 trunc from 0xb342200 to 0x9a52400 2024-09-19T21:35:34.902 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:34.902 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:34.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:34.929 INFO:teuthology.orchestra.run.smithi122.stdout:3025 write 0xb2f4e00 thru 0xb2fc734 (0x7935 bytes) 2024-09-19T21:35:34.931 INFO:teuthology.orchestra.run.smithi122.stdout:3026 write 0xa096c00 thru 0xa09b91b (0x4d1c bytes) 2024-09-19T21:35:34.933 INFO:teuthology.orchestra.run.smithi122.stdout:3027 punch from 0xa8a2a00 to 0xa8ade00, (0xb400 bytes) 2024-09-19T21:35:34.934 INFO:teuthology.orchestra.run.smithi122.stdout:3028 read 0x5c71600 thru 0x5c7e375 (0xcd76 bytes) 2024-09-19T21:35:34.938 INFO:teuthology.orchestra.run.smithi122.stdout:3029 write 0x34b4000 thru 0x34b7f64 (0x3f65 bytes) 2024-09-19T21:35:34.939 INFO:teuthology.orchestra.run.smithi122.stdout:3031 write 0x4658a00 thru 0x4662a6b (0xa06c bytes) 2024-09-19T21:35:34.940 INFO:teuthology.orchestra.run.smithi122.stdout:3035 read 0x659a800 thru 0x65a35f6 (0x8df7 bytes) 2024-09-19T21:35:34.941 INFO:teuthology.orchestra.run.smithi122.stdout:3038 write 0xa291e00 thru 0xa29c7ed (0xa9ee bytes) 2024-09-19T21:35:34.942 INFO:teuthology.orchestra.run.smithi122.stdout:3042 read 0x29bde00 thru 0x29cd9a2 (0xfba3 bytes) 2024-09-19T21:35:34.945 INFO:teuthology.orchestra.run.smithi122.stdout:3043 punch from 0x8842800 to 0x8846000, (0x3800 bytes) 2024-09-19T21:35:34.946 INFO:teuthology.orchestra.run.smithi122.stdout:3045 punch from 0xa92ca00 to 0xa92da00, (0x1000 bytes) 2024-09-19T21:35:34.946 INFO:teuthology.orchestra.run.smithi122.stdout:3046 write 0xf54e00 thru 0xf640df (0xf2e0 bytes) 2024-09-19T21:35:34.949 INFO:teuthology.orchestra.run.smithi122.stdout:3049 read 0x8421000 thru 0x8421d3a (0xd3b bytes) 2024-09-19T21:35:34.962 INFO:teuthology.orchestra.run.smithi122.stdout:3050 trunc from 0xb2fc800 to 0x5e79000 2024-09-19T21:35:34.965 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:34.962+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:34.999 INFO:teuthology.orchestra.run.smithi122.stdout:3051 read 0x5a06400 thru 0x5a0c514 (0x6115 bytes) 2024-09-19T21:35:35.002 INFO:teuthology.orchestra.run.smithi122.stdout:3053 read 0x1e2c600 thru 0x1e325f2 (0x5ff3 bytes) 2024-09-19T21:35:35.003 INFO:teuthology.orchestra.run.smithi122.stdout:3054 read 0x5560200 thru 0x5560d5e (0xb5f bytes) 2024-09-19T21:35:35.005 INFO:teuthology.orchestra.run.smithi122.stdout:3058 punch from 0x2128000 to 0x212d200, (0x5200 bytes) 2024-09-19T21:35:35.005 INFO:teuthology.orchestra.run.smithi122.stdout:3059 read 0xcaa400 thru 0xcb462f (0xa230 bytes) 2024-09-19T21:35:35.008 INFO:teuthology.orchestra.run.smithi122.stdout:3061 punch from 0x130c00 to 0x13c800, (0xbc00 bytes) 2024-09-19T21:35:35.009 INFO:teuthology.orchestra.run.smithi122.stdout:3062 flatten 2024-09-19T21:35:35.034 INFO:teuthology.orchestra.run.smithi122.stdout:3064 punch from 0x1469600 to 0x146d000, (0x3a00 bytes) 2024-09-19T21:35:35.066 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:35.062+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:35.068 INFO:teuthology.orchestra.run.smithi122.stdout:3067 write 0x9e4f200 thru 0x9e4f754 (0x555 bytes) 2024-09-19T21:35:35.070 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:35.070 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:35.070 INFO:teuthology.orchestra.run.smithi122.stdout:3071 read 0x82a6400 thru 0x82b2bb9 (0xc7ba bytes) 2024-09-19T21:35:35.071 INFO:teuthology.orchestra.run.smithi122.stdout:3073 read 0x7206c00 thru 0x7215b4d (0xef4e bytes) 2024-09-19T21:35:35.073 INFO:teuthology.orchestra.run.smithi122.stdout:3074 read 0x7863600 thru 0x78663ed (0x2dee bytes) 2024-09-19T21:35:35.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:35.113 INFO:teuthology.orchestra.run.smithi122.stdout:3077 write 0xe8f6c00 thru 0xe903b85 (0xcf86 bytes) 2024-09-19T21:35:35.115 INFO:teuthology.orchestra.run.smithi122.stdout:3079 write 0x9a90c00 thru 0x9a956bb (0x4abc bytes) 2024-09-19T21:35:35.116 INFO:teuthology.orchestra.run.smithi122.stdout:3082 punch from 0x8a1a00 to 0x8a7c00, (0x6200 bytes) 2024-09-19T21:35:35.117 INFO:teuthology.orchestra.run.smithi122.stdout:3084 write 0x8ee5600 thru 0x8eed3f3 (0x7df4 bytes) 2024-09-19T21:35:35.118 INFO:teuthology.orchestra.run.smithi122.stdout:3085 write 0xe6b8e00 thru 0xe6c052d (0x772e bytes) 2024-09-19T21:35:35.119 INFO:teuthology.orchestra.run.smithi122.stdout:3087 read 0xa7b7e00 thru 0xa7b8ac4 (0xcc5 bytes) 2024-09-19T21:35:35.120 INFO:teuthology.orchestra.run.smithi122.stdout:3088 read 0x6d08000 thru 0x6d0853b (0x53c bytes) 2024-09-19T21:35:35.121 INFO:teuthology.orchestra.run.smithi122.stdout:3089 read 0x34da800 thru 0x34dd7ba (0x2fbb bytes) 2024-09-19T21:35:35.122 INFO:teuthology.orchestra.run.smithi122.stdout:3091 punch from 0xe300a00 to 0xe308200, (0x7800 bytes) 2024-09-19T21:35:35.133 INFO:teuthology.orchestra.run.smithi122.stdout:3095 trunc from 0xe903c00 to 0xa69e600 2024-09-19T21:35:35.164 INFO:teuthology.orchestra.run.smithi122.stdout:3097 read 0x5b70800 thru 0x5b7825c (0x7a5d bytes) 2024-09-19T21:35:35.166 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:35.162+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:35.167 INFO:teuthology.orchestra.run.smithi122.stdout:3098 trunc from 0xa69e600 to 0xaaf9800 2024-09-19T21:35:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:35.196 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:35.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:35.214 INFO:teuthology.orchestra.run.smithi122.stdout:3101 write 0xb32a600 thru 0xb32dcf2 (0x36f3 bytes) 2024-09-19T21:35:35.242 INFO:teuthology.orchestra.run.smithi122.stdout:3102 write 0xba0d600 thru 0xba112b6 (0x3cb7 bytes) 2024-09-19T21:35:35.245 INFO:teuthology.orchestra.run.smithi122.stdout:3103 write 0x15c0800 thru 0x15c6ee3 (0x66e4 bytes) 2024-09-19T21:35:35.246 INFO:teuthology.orchestra.run.smithi122.stdout:3104 write 0x6af9a00 thru 0x6afce3a (0x343b bytes) 2024-09-19T21:35:35.247 INFO:teuthology.orchestra.run.smithi122.stdout:3106 write 0x3ec2e00 thru 0x3ec9762 (0x6963 bytes) 2024-09-19T21:35:35.247 INFO:teuthology.orchestra.run.smithi122.stdout:3107 read 0x4c4f400 thru 0x4c5e4d5 (0xf0d6 bytes) 2024-09-19T21:35:35.248 INFO:teuthology.orchestra.run.smithi122.stdout:3108 read 0x1625000 thru 0x1632ddf (0xdde0 bytes) 2024-09-19T21:35:35.252 INFO:teuthology.orchestra.run.smithi122.stdout:3111 trunc from 0xba11400 to 0xc719e00 2024-09-19T21:35:35.267 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:35.266+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:35.299 INFO:teuthology.orchestra.run.smithi122.stdout:3112 trunc from 0xc719e00 to 0x1f87200 2024-09-19T21:35:35.313 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:35.313 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:35.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:35.368 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:35.360+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:35.402 INFO:teuthology.orchestra.run.smithi122.stdout:3113 write 0xba0ea00 thru 0xba1973d (0xad3e bytes) 2024-09-19T21:35:35.403 INFO:teuthology.orchestra.run.smithi122.stdout:3114 clone 21 order 18 su 1024 sc 13 2024-09-19T21:35:35.445 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:35.445 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:35.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:35.468 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:35.466+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:35.557 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:35.560 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:35.560 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:35.569 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:35.564+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:35.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:35.670 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:35.666+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:35.690 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:35.711 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:35.724 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:35.724 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:35.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.3634296360000002, 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:35.729 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.3634173280000002, 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:35.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:35.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.733 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.3306860889999998, 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:35.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.100928396, 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:35.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1009049609999999, 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:35.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:35.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:35.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1008832329999998, 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:35.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1008625009999999, 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.754 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:35.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:35.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:35.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:35.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:35.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:35.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.100828065, 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:35.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.1007852680000001, 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:35.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0955597579999998, 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:35.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:35.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:35.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.777 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0955374600000001, 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:35.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:35.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:35.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.782 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0955066050000002, 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:35.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:35.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:35.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:35.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.095463896, 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:35.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:35.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.792 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.095439909, 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:35.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:35.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.095408505, 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.797 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.801 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0953833390000001, 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:35.805 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0953382610000002, 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:35.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0953115169999998, 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:35.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.0952982250000001, 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.815 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:35.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:35.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.095281306, 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:35.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:35.822 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:35.823 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:35.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:35.825 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:35.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:35.827 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:35.816+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:35.924 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:35.920+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:35.981 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:35.981 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:35.982 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:35.982 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:35.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:36.026 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:36.020+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:36.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:35:36.124 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:36.124 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:36.124 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:36.124 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:36.125 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:36.127 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:36.122+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:36.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:36.228 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:36.224+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:36.244 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 80.307456748000007, 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9878894169999999, 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:35:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9878776509999998, 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:35:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9878647579999997, 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9878527350000001, 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:35:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.987835091, 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:35:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:35:36.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.275 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.987819295, 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.987807444, 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:35:36.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:35:36.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9877386000000001, 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:35:36.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:35:36.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.289 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9877104599999997, 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:35:36.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9876873310000001, 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:35:36.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9729107109999999, 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:35:36.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:35:36.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9655743960000001, 2024-09-19T21:35:36.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:35:36.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:35:36.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.965553506, 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:36.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.312 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9584779660000002, 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:35:36.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9584574219999999, 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.317 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:36.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:35:36.321 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7781033509999999, 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:36.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:35:36.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.326 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7780929240000001, 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:35:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.778082495, 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:35:36.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:36.335 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7780724320000001, 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:35:36.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:35:36.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:35:36.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.340 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.340 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.340 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.340 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.340 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:36.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:36.342 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:36.336+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:36.443 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:36.438+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:36.447 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:36.447 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:36.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:36.543 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:36.542+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:36.565 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:36.565 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:36.565 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:36.566 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:36.566 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:36.576 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:36.644 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:36.642+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.735 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.503574030999999, 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.736 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:36.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:36.740 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.029546222999997, 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.029516395999998, 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:36.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:36.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:36.750 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.02950671, 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:36.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:36.755 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.029498384, 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:36.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:36.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.759 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.029487992, 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.029477495999998, 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.769 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.027033408000001, 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.001816022, 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:36.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:36.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15066 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6164480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106221331, 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029164998000000001, 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15066 2024-09-19T21:35:36.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624793+0000", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624797+0000", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4580000000000002e-06 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624803+0000", 2024-09-19T21:35:36.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6800000000000004e-06 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624809+0000", 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8969999999999996e-06 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653805+0000", 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028996141999999999 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653889+0000", 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3906000000000006e-05 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653959+0000", 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9604000000000006e-05 2024-09-19T21:35:36.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15067 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6178816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106192721999999, 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029307910999999999, 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15067 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624831+0000", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2019999999999993e-06 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624840+0000", 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5639999999999997e-06 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624846+0000", 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4740000000000002e-06 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653998+0000", 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029151795000000001 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654062+0000", 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4289999999999996e-05 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654130+0000", 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8034000000000006e-05 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15068 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6193152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106164720999999, 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029394144000000001, 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15068 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624853+0000", 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8339999999999999e-06 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624860+0000", 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7689999999999999e-06 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624866+0000", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0639999999999999e-06 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654143+0000", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277609999999999 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654189+0000", 2024-09-19T21:35:36.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5892e-05 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654245+0000", 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5383000000000002e-05 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15069 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106144375, 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029542570000000001, 2024-09-19T21:35:36.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15069 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624870+0000", 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624873+0000", 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624880+0000", 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624886+0000", 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654272+0000", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029385674000000001 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654328+0000", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6156999999999998e-05 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654413+0000", 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999998e-05 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.802 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15070 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106124186999999, 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02965835, 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15070 2024-09-19T21:35:36.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624894+0000", 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9900000000000002e-06 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624903+0000", 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1220000000000001e-06 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624917+0000", 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3973e-05 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654426+0000", 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029508869 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654475+0000", 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9274999999999998e-05 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654549+0000", 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4492000000000004e-05 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15071 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6236160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.807 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106092153999999, 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029922358, 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15071 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624922+0000", 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624926+0000", 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2650000000000001e-06 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624933+0000", 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624939+0000", 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9819999999999996e-06 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654596+0000", 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029657246000000002 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654708+0000", 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112194 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654845+0000", 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013685999999999999 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15072 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6250496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106070742, 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030057148999999998, 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15072 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624947+0000", 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.326e-06 2024-09-19T21:35:36.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624954+0000", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6590000000000001e-06 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624959+0000", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3630000000000002e-06 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654863+0000", 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029903855999999999 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654917+0000", 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4487000000000002e-05 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655002+0000", 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4300000000000003e-05 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.816 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15073 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6264832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106049738999999, 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030212118999999999, 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15073 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624964+0000", 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:36.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624968+0000", 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.16e-06 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624975+0000", 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9600000000000003e-06 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624981+0000", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7980000000000002e-06 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655037+0000", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030056058 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655093+0000", 2024-09-19T21:35:36.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6223e-05 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655177+0000", 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4654000000000006e-05 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15074 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.106029513999999, 2024-09-19T21:35:36.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030446608, 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15074 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:36.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624985+0000", 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624988+0000", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1370000000000002e-06 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625010+0000", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1778000000000001e-05 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625017+0000", 2024-09-19T21:35:36.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5259999999999999e-06 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655204+0000", 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030186983000000001 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655303+0000", 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:36.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655432+0000", 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012875899999999999 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18639 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 731136~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.709052043, 2024-09-19T21:35:36.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029303412000000001, 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18639 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021964+0000", 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021967+0000", 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021970+0000", 2024-09-19T21:35:36.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2349999999999999e-06 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051070+0000", 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029100002 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051159+0000", 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9033e-05 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051266+0000", 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107069 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18640 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 745472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.709031672999998, 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029524404000000001, 2024-09-19T21:35:36.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18640 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021985+0000", 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9750000000000001e-06 2024-09-19T21:35:36.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021988+0000", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9720000000000001e-06 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021991+0000", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6369999999999999e-06 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051317+0000", 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029326609 2024-09-19T21:35:36.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051398+0000", 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0705999999999996e-05 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051508+0000", 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109948 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:36.835 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:36.837 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-19T21:35:36.837 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:36.832+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:36.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:36.937 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:36.934+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:36.964 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:36.965 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:36.965 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:36.965 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:36.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:37.038 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:37.032+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:37.038 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:35:37.108 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.108 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:37.109 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:37.109 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:37.139 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:37.134+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979270790000001, 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:37.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979257251, 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979231511000002, 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:37.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979217297000002, 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979196380000001, 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:37.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979181004000001, 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979166075999998, 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:37.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979113456, 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:37.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979063878000002, 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.276 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:37.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979046662999998, 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:37.285 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979027524999999, 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.979012677, 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:37.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.978989683999998, 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:37.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.978257392, 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.978207082000001, 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.97814477, 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.976814137000002, 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.976678559, 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.323 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.976601800000001, 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.976543075999999, 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:37.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.332 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.333 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.333 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.334 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:37.335 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:37.328+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:37.437 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:37.428+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:37.470 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.470 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:37.470 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:37.470 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:37.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:37.528+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:37.607 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.607 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:37.607 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:37.607 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:37.607 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:37.637 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:37.634+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4082 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5201536050000009, 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068451995000000002, 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4082 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196961+0000", 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196962+0000", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.734e-06 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196965+0000", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6979999999999998e-06 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196972+0000", 2024-09-19T21:35:37.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9369999999999998e-06 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.264911+0000", 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067938800999999993 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.264995+0000", 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4637999999999998e-05 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265413+0000", 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041739199999999999 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4083 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 688128~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5201401519999997, 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068703602000000003, 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4083 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196974+0000", 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196976+0000", 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8470000000000001e-06 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196979+0000", 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.038e-06 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.196985+0000", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3539999999999996e-06 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265450+0000", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068464982999999993 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265554+0000", 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104085 2024-09-19T21:35:37.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265678+0000", 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000123502 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4095 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5195912660000008, 2024-09-19T21:35:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068425451999999998, 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4095 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197523+0000", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197526+0000", 2024-09-19T21:35:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3139999999999999e-06 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197532+0000", 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4250000000000003e-06 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197535+0000", 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1300000000000001e-06 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265709+0000", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068173553999999997 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265800+0000", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.1246e-05 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265949+0000", 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000148317 2024-09-19T21:35:37.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4097 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5195760109999998, 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068888300999999999, 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4097 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197538+0000", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197540+0000", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.283e-06 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197544+0000", 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.083e-06 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197547+0000", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.005e-06 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.265987+0000", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068440300999999995 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266046+0000", 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8983000000000001e-05 2024-09-19T21:35:37.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266427+0000", 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00038087700000000001 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4099 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5195653720000006, 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069491868999999998, 2024-09-19T21:35:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4099 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197549+0000", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197567+0000", 2024-09-19T21:35:37.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8372999999999998e-05 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197576+0000", 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2989999999999997e-06 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197581+0000", 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8050000000000003e-06 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266471+0000", 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.068890122999999998 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.266606+0000", 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013495799999999999 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267041+0000", 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00043451999999999998 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4101 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5195274059999999, 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069876111000000005, 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4101 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197587+0000", 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197586+0000", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197589+0000", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0790000000000001e-06 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197599+0000", 2024-09-19T21:35:37.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6039999999999999e-06 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197605+0000", 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4920000000000001e-06 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267096+0000", 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069491154999999999 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267162+0000", 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6161000000000006e-05 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267463+0000", 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000301238 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.749 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4109 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5195052830000009, 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070017945999999998, 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4109 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197609+0000", 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197611+0000", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.712e-06 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197621+0000", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4250000000000004e-06 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197627+0000", 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.7259999999999999e-06 2024-09-19T21:35:37.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267488+0000", 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069861147999999998 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267547+0000", 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9497000000000003e-05 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267627+0000", 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9831000000000002e-05 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5194212789999995, 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:35:37.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:35:37.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5188413680000004, 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:35:37.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5187787210000003, 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:35:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5187124010000002, 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:35:37.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5180087229999994, 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:35:37.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:35:37.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5179359009999995, 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.782 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5179221629999997, 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:37.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:35:37.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5179044739999998, 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.517854088, 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5178271989999992, 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:35:37.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:35:37.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5177866229999992, 2024-09-19T21:35:37.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:35:37.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5177650089999997, 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:35:37.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:35:37.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.5177458099999992, 2024-09-19T21:35:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:35:37.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:37.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.814 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:37.815 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:37.815 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:37.817 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:37.812+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:37.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:37.878 INFO:teuthology.orchestra.run.smithi122.stdout:3119 write 0x9bc5000 thru 0x9bd27e8 (0xd7e9 bytes) 2024-09-19T21:35:37.880 INFO:teuthology.orchestra.run.smithi122.stdout:3120 write 0x4053800 thru 0x405c219 (0x8a1a bytes) 2024-09-19T21:35:37.882 INFO:teuthology.orchestra.run.smithi122.stdout:3122 punch from 0xb983600 to 0xb98c200, (0x8c00 bytes) 2024-09-19T21:35:37.917 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:37.912+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:37.933 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:37.934 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:37.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:37.984 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:38.007 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:38.018 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:38.014+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:38.026 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:38.045 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:38.066 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:38.090 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:38.092 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:38.092 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:38.092 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:38.092 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:38.092 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:38.093 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:38.106 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.106 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.118 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:38.119 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:38.118+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:38.220 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:38.216+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:38.225 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.225 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:38.321 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:38.318+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:38.360 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.360 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:38.422 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:38.418+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:38.522 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:38.516+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:38.528 INFO:teuthology.orchestra.run.smithi122.stdout:3124 read 0xac4b800 thru 0xac5a22c (0xea2d bytes) 2024-09-19T21:35:38.549 INFO:teuthology.orchestra.run.smithi122.stdout:3125 write 0x7b71400 thru 0x7b76a92 (0x5693 bytes) 2024-09-19T21:35:38.550 INFO:teuthology.orchestra.run.smithi122.stdout:3126 punch from 0x6e58000 to 0x6e64800, (0xc800 bytes) 2024-09-19T21:35:38.605 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.605 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:38.659 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:38.658+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:38.724 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:38.722+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:38.731 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.731 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.742 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:38.825 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:38.822+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:38.846 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.847 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:38.926 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:38.920+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:38.963 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:38.963 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:38.973 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:39.027 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:39.020+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:39.042 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:39.042 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:39.042 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T21:35:39.042 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:35:39.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:35:39.081 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.082 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:39.135 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:39.134+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:39.229 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:39.224+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:39.269 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.269 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:39.330 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:39.326+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:39.418 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.419 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:39.471 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:39.470+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 65, 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:35:37.027803+0000", 2024-09-19T21:35:39.503 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:34:04.640393+0000", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5804032, 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "nodeep-scrub", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:35:39.504 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "luminous", 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:35:39.505 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:39.506 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:39.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:39.508 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:39.509 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:39.510 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:35:39.511 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:39.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:35:39.513 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.514 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:39.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:39.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:39.517 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.25, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.25, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:35:39.519 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:35:39.520 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "64", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 24, 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 64, 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:35:39.521 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:35:39.522 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:35:39.523 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.1299999952316284, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.1299999952316284, 2024-09-19T21:35:39.524 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 1, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:35:39.525 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 56, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:35:39.530 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.531 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:35:39.533 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.535 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:39.537 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.541 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:35:39.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 24, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.544 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.546 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.550 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:35:39.556 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:35:39.560 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:35:39.561 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.562 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.564 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.565 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:35:39.566 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [ 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "pgid": "3.15", 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "mappings": [ 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "from": 3, 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: "to": 5 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:39.567 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:35:39.568 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:35:39.569 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:35:39.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:39.570 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:39.572 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.572 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.573 INFO:tasks.thrashosds.thrasher:j is {'epoch': 65, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:35:37.027803+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:34:04.640393+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.15', 'mappings': [{'from': 3, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:35:39.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:35:39.573 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:39.566+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:39.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:39.638 INFO:teuthology.orchestra.run.smithi122.stdout:3129 read 0xad89200 thru 0xad8b742 (0x2543 bytes) 2024-09-19T21:35:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:3131 punch from 0xa3b9200 to 0xa3bb200, (0x2000 bytes) 2024-09-19T21:35:39.672 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:39.670+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:39.711 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.711 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:39.772 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:39.770+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:39.845 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.845 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:39.872 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:39.868+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:35:39.894 INFO:teuthology.orchestra.run.smithi122.stdout:3132 punch from 0x5061e00 to 0x506ec00, (0xce00 bytes) 2024-09-19T21:35:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:35:39.908 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-19T21:35:38.820045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1177634304,"num_objects":1017,"num_object_clones":279,"num_object_copies":2034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1017,"num_whiteouts":276,"num_read":170078,"num_read_kb":705943,"num_write":8807,"num_write_kb":170634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":196811,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7817,"ondisk_log_size":7817,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":491188,"kb_used_data":259148,"kb_used_omap":9,"kb_used_meta":231670,"kb_avail":565739852,"statfs":{"total":579820584960,"available":579317608448,"internally_reserved":0,"allocated":265367552,"data_stored":340822064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":237230758},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,61,81,103],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":108910311,"num_objects":228,"num_object_clones":62,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":18,"num_read":20055,"num_read_kb":137997,"num_write":1130,"num_write_kb":52298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001685"},"pg_stats":[{"pgid":"3.1f","version":"62'246","reported_seq":2412,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.069346+0000","last_change":"2024-09-19T21:34:32.012011+0000","last_active":"2024-09-19T21:35:37.069346+0000","last_peered":"2024-09-19T21:35:37.069346+0000","last_clean":"2024-09-19T21:35:37.069346+0000","last_became_active":"2024-09-19T21:34:30.013688+0000","last_became_peered":"2024-09-19T21:34:30.013688+0000","last_unstale":"2024-09-19T21:35:37.069346+0000","last_undegraded":"2024-09-19T21:35:37.069346+0000","last_fullsized":"2024-09-19T21:35:37.069346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:27:13.050028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":40878126,"num_objects":39,"num_object_clones":11,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":7,"num_read":3758,"num_read_kb":31874,"num_write":240,"num_write_kb":21363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"62'100","reported_seq":684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.151287+0000","last_change":"2024-09-19T21:34:32.006487+0000","last_active":"2024-09-19T21:35:34.151287+0000","last_peered":"2024-09-19T21:35:34.151287+0000","last_clean":"2024-09-19T21:35:34.151287+0000","last_became_active":"2024-09-19T21:34:30.011721+0000","last_became_peered":"2024-09-19T21:34:30.011721+0000","last_unstale":"2024-09-19T21:35:34.151287+0000","last_undegraded":"2024-09-19T21:35:34.151287+0000","last_fullsized":"2024-09-19T21:35:34.151287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:21.923196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":12699700,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":584,"num_read_kb":4124,"num_write":77,"num_write_kb":1178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"64'634","reported_seq":2798,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.678887+0000","last_change":"2024-09-19T21:34:32.024269+0000","last_active":"2024-09-19T21:35:37.678887+0000","last_peered":"2024-09-19T21:35:37.678887+0000","last_clean":"2024-09-19T21:35:37.678887+0000","last_became_active":"2024-09-19T21:34:30.010774+0000","last_became_peered":"2024-09-19T21:34:30.010774+0000","last_unstale":"2024-09-19T21:35:37.678887+0000","last_undegraded":"2024-09-19T21:35:37.678887+0000","last_fullsized":"2024-09-19T21:35:37.678887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:59.647828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":58945229,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":4,"num_read":5058,"num_read_kb":23392,"num_write":1075,"num_write_kb":12295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"64'280","reported_seq":11637,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.685198+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:35:37.685198+0000","last_peered":"2024-09-19T21:35:37.685198+0000","last_clean":"2024-09-19T21:35:37.685198+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:35:37.685198+0000","last_undegraded":"2024-09-19T21:35:37.685198+0000","last_fullsized":"2024-09-19T21:35:37.685198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":13811764,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":12590,"num_read_kb":20105,"num_write":227,"num_write_kb":1670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"61'242","reported_seq":3313,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.151826+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:35:34.151826+0000","last_peered":"2024-09-19T21:35:34.151826+0000","last_clean":"2024-09-19T21:35:34.151826+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:35:34.151826+0000","last_undegraded":"2024-09-19T21:35:34.151826+0000","last_fullsized":"2024-09-19T21:35:34.151826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":121388179,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":9,"num_read":3219,"num_read_kb":24251,"num_write":239,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"65'355","reported_seq":4005,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.212272+0000","last_change":"2024-09-19T21:34:32.001087+0000","last_active":"2024-09-19T21:35:38.212272+0000","last_peered":"2024-09-19T21:35:38.212272+0000","last_clean":"2024-09-19T21:35:38.212272+0000","last_became_active":"2024-09-19T21:34:30.002669+0000","last_became_peered":"2024-09-19T21:34:30.002669+0000","last_unstale":"2024-09-19T21:35:38.212272+0000","last_undegraded":"2024-09-19T21:35:38.212272+0000","last_fullsized":"2024-09-19T21:35:38.212272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:54:28.569315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":17672528,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":4938,"num_read_kb":30021,"num_write":541,"num_write_kb":4234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"65'274","reported_seq":2441,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.035327+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:35:37.035327+0000","last_peered":"2024-09-19T21:35:37.035327+0000","last_clean":"2024-09-19T21:35:37.035327+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:35:37.035327+0000","last_undegraded":"2024-09-19T21:35:37.035327+0000","last_fullsized":"2024-09-19T21:35:37.035327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":27852978,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":2967,"num_read_kb":17313,"num_write":370,"num_write_kb":7556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"63'169","reported_seq":5021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:35.324578+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:35:35.324578+0000","last_peered":"2024-09-19T21:35:35.324578+0000","last_clean":"2024-09-19T21:35:35.324578+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:35:35.324578+0000","last_undegraded":"2024-09-19T21:35:35.324578+0000","last_fullsized":"2024-09-19T21:35:35.324578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":32055358,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":7,"num_read":5626,"num_read_kb":21722,"num_write":196,"num_write_kb":6001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"62'127","reported_seq":4208,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.434921+0000","last_change":"2024-09-19T21:34:32.012032+0000","last_active":"2024-09-19T21:35:37.434921+0000","last_peered":"2024-09-19T21:35:37.434921+0000","last_clean":"2024-09-19T21:35:37.434921+0000","last_became_active":"2024-09-19T21:34:30.011066+0000","last_became_peered":"2024-09-19T21:34:30.011066+0000","last_unstale":"2024-09-19T21:35:37.434921+0000","last_undegraded":"2024-09-19T21:35:37.434921+0000","last_fullsized":"2024-09-19T21:35:37.434921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:40:19.121173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":8425984,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":4067,"num_read_kb":8802,"num_write":106,"num_write_kb":1866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"62'136","reported_seq":8492,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.991937+0000","last_change":"2024-09-19T21:34:32.024453+0000","last_active":"2024-09-19T21:35:36.991937+0000","last_peered":"2024-09-19T21:35:36.991937+0000","last_clean":"2024-09-19T21:35:36.991937+0000","last_became_active":"2024-09-19T21:34:30.006875+0000","last_became_peered":"2024-09-19T21:34:30.006875+0000","last_unstale":"2024-09-19T21:35:36.991937+0000","last_undegraded":"2024-09-19T21:35:36.991937+0000","last_fullsized":"2024-09-19T21:35:36.991937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:26:13.507158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":37753384,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":8379,"num_read_kb":25380,"num_write":132,"num_write_kb":3395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"65'100","reported_seq":4886,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.072044+0000","last_change":"2024-09-19T21:35:24.560989+0000","last_active":"2024-09-19T21:35:38.072044+0000","last_peered":"2024-09-19T21:35:38.072044+0000","last_clean":"2024-09-19T21:35:38.072044+0000","last_became_active":"2024-09-19T21:35:24.463594+0000","last_became_peered":"2024-09-19T21:35:24.463594+0000","last_unstale":"2024-09-19T21:35:38.072044+0000","last_undegraded":"2024-09-19T21:35:38.072044+0000","last_fullsized":"2024-09-19T21:35:38.072044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:16.658378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":11162315,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":4714,"num_read_kb":7875,"num_write":82,"num_write_kb":673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":196811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"62'146","reported_seq":7954,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.999662+0000","last_change":"2024-09-19T21:34:32.002842+0000","last_active":"2024-09-19T21:35:36.999662+0000","last_peered":"2024-09-19T21:35:36.999662+0000","last_clean":"2024-09-19T21:35:36.999662+0000","last_became_active":"2024-09-19T21:34:30.010116+0000","last_became_peered":"2024-09-19T21:34:30.010116+0000","last_unstale":"2024-09-19T21:35:36.999662+0000","last_undegraded":"2024-09-19T21:35:36.999662+0000","last_fullsized":"2024-09-19T21:35:36.999662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:38:12.596445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":34860560,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":12,"num_read":7850,"num_read_kb":22819,"num_write":142,"num_write_kb":3583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.024243+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:35:36.024243+0000","last_peered":"2024-09-19T21:35:36.024243+0000","last_clean":"2024-09-19T21:35:36.024243+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:35:36.024243+0000","last_undegraded":"2024-09-19T21:35:36.024243+0000","last_fullsized":"2024-09-19T21:35:36.024243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"62'264","reported_seq":6509,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.951986+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:35:36.951986+0000","last_peered":"2024-09-19T21:35:36.951986+0000","last_clean":"2024-09-19T21:35:36.951986+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:35:36.951986+0000","last_undegraded":"2024-09-19T21:35:36.951986+0000","last_fullsized":"2024-09-19T21:35:36.951986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":50846797,"num_objects":31,"num_object_clones":10,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":6665,"num_read_kb":16130,"num_write":247,"num_write_kb":4460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.034955+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:35:36.034955+0000","last_peered":"2024-09-19T21:35:36.034955+0000","last_clean":"2024-09-19T21:35:36.034955+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:35:36.034955+0000","last_undegraded":"2024-09-19T21:35:36.034955+0000","last_fullsized":"2024-09-19T21:35:36.034955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"62'398","reported_seq":3820,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.530059+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:35:37.530059+0000","last_peered":"2024-09-19T21:35:37.530059+0000","last_clean":"2024-09-19T21:35:37.530059+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:35:37.530059+0000","last_undegraded":"2024-09-19T21:35:37.530059+0000","last_fullsized":"2024-09-19T21:35:37.530059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":18765889,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":5244,"num_read_kb":10215,"num_write":485,"num_write_kb":2708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.023485+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:35:36.023485+0000","last_peered":"2024-09-19T21:35:36.023485+0000","last_clean":"2024-09-19T21:35:36.023485+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:35:36.023485+0000","last_undegraded":"2024-09-19T21:35:36.023485+0000","last_fullsized":"2024-09-19T21:35:36.023485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"65'148","reported_seq":8598,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.680171+0000","last_change":"2024-09-19T21:34:32.024327+0000","last_active":"2024-09-19T21:35:37.680171+0000","last_peered":"2024-09-19T21:35:37.680171+0000","last_clean":"2024-09-19T21:35:37.680171+0000","last_became_active":"2024-09-19T21:34:30.007041+0000","last_became_peered":"2024-09-19T21:34:30.007041+0000","last_unstale":"2024-09-19T21:35:37.680171+0000","last_undegraded":"2024-09-19T21:35:37.680171+0000","last_fullsized":"2024-09-19T21:35:37.680171+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:31:26.974439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":37609509,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":8454,"num_read_kb":36885,"num_write":130,"num_write_kb":4884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.013607+0000","last_change":"2024-09-19T21:34:31.005184+0000","last_active":"2024-09-19T21:35:34.013607+0000","last_peered":"2024-09-19T21:35:34.013607+0000","last_clean":"2024-09-19T21:35:34.013607+0000","last_became_active":"2024-09-19T21:34:31.004633+0000","last_became_peered":"2024-09-19T21:34:31.004633+0000","last_unstale":"2024-09-19T21:35:34.013607+0000","last_undegraded":"2024-09-19T21:35:34.013607+0000","last_fullsized":"2024-09-19T21:35:34.013607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:21.400292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"61'337","reported_seq":3546,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.999129+0000","last_change":"2024-09-19T21:34:32.024762+0000","last_active":"2024-09-19T21:35:36.999129+0000","last_peered":"2024-09-19T21:35:36.999129+0000","last_clean":"2024-09-19T21:35:36.999129+0000","last_became_active":"2024-09-19T21:34:30.006903+0000","last_became_peered":"2024-09-19T21:34:30.006903+0000","last_unstale":"2024-09-19T21:35:36.999129+0000","last_undegraded":"2024-09-19T21:35:36.999129+0000","last_fullsized":"2024-09-19T21:35:36.999129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.226419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":39328328,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":5541,"num_read_kb":18119,"num_write":380,"num_write_kb":5942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"65'566","reported_seq":5489,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.172814+0000","last_change":"2024-09-19T21:34:32.006802+0000","last_active":"2024-09-19T21:35:38.172814+0000","last_peered":"2024-09-19T21:35:38.172814+0000","last_clean":"2024-09-19T21:35:38.172814+0000","last_became_active":"2024-09-19T21:34:30.011547+0000","last_became_peered":"2024-09-19T21:34:30.011547+0000","last_unstale":"2024-09-19T21:35:38.172814+0000","last_undegraded":"2024-09-19T21:35:38.172814+0000","last_fullsized":"2024-09-19T21:35:38.172814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:45:47.281981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":25119013,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":9751,"num_read_kb":13142,"num_write":637,"num_write_kb":2855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":62,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.040669+0000","last_change":"2024-09-19T21:34:22.954354+0000","last_active":"2024-09-19T21:35:36.040669+0000","last_peered":"2024-09-19T21:35:36.040669+0000","last_clean":"2024-09-19T21:35:36.040669+0000","last_became_active":"2024-09-19T21:34:20.956623+0000","last_became_peered":"2024-09-19T21:34:20.956623+0000","last_unstale":"2024-09-19T21:35:36.040669+0000","last_undegraded":"2024-09-19T21:35:36.040669+0000","last_fullsized":"2024-09-19T21:35:36.040669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:00:52.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"63'165","reported_seq":2696,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.678715+0000","last_change":"2024-09-19T21:34:32.011746+0000","last_active":"2024-09-19T21:35:37.678715+0000","last_peered":"2024-09-19T21:35:37.678715+0000","last_clean":"2024-09-19T21:35:37.678715+0000","last_became_active":"2024-09-19T21:34:30.013347+0000","last_became_peered":"2024-09-19T21:34:30.013347+0000","last_unstale":"2024-09-19T21:35:37.678715+0000","last_undegraded":"2024-09-19T21:35:37.678715+0000","last_fullsized":"2024-09-19T21:35:37.678715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.129809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":26369604,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":3554,"num_read_kb":31292,"num_write":145,"num_write_kb":7478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.013068+0000","last_change":"2024-09-19T21:34:23.705148+0000","last_active":"2024-09-19T21:35:34.013068+0000","last_peered":"2024-09-19T21:35:34.013068+0000","last_clean":"2024-09-19T21:35:34.013068+0000","last_became_active":"2024-09-19T21:34:20.962501+0000","last_became_peered":"2024-09-19T21:34:20.962501+0000","last_unstale":"2024-09-19T21:35:34.013068+0000","last_undegraded":"2024-09-19T21:35:34.013068+0000","last_fullsized":"2024-09-19T21:35:34.013068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:52.808827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"65'272","reported_seq":3018,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.173523+0000","last_change":"2024-09-19T21:34:32.024303+0000","last_active":"2024-09-19T21:35:38.173523+0000","last_peered":"2024-09-19T21:35:38.173523+0000","last_clean":"2024-09-19T21:35:38.173523+0000","last_became_active":"2024-09-19T21:34:30.009730+0000","last_became_peered":"2024-09-19T21:34:30.009730+0000","last_unstale":"2024-09-19T21:35:38.173523+0000","last_undegraded":"2024-09-19T21:35:38.173523+0000","last_fullsized":"2024-09-19T21:35:38.173523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.487727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":56677413,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":3925,"num_read_kb":19364,"num_write":367,"num_write_kb":6178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.024484+0000","last_change":"2024-09-19T21:34:23.702606+0000","last_active":"2024-09-19T21:35:36.024484+0000","last_peered":"2024-09-19T21:35:36.024484+0000","last_clean":"2024-09-19T21:35:36.024484+0000","last_became_active":"2024-09-19T21:34:20.954829+0000","last_became_peered":"2024-09-19T21:34:20.954829+0000","last_unstale":"2024-09-19T21:35:36.024484+0000","last_undegraded":"2024-09-19T21:35:36.024484+0000","last_fullsized":"2024-09-19T21:35:36.024484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:48:58.350629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"61'250","reported_seq":9017,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.151726+0000","last_change":"2024-09-19T21:34:32.005130+0000","last_active":"2024-09-19T21:35:34.151726+0000","last_peered":"2024-09-19T21:35:34.151726+0000","last_clean":"2024-09-19T21:35:34.151726+0000","last_became_active":"2024-09-19T21:34:30.009669+0000","last_became_peered":"2024-09-19T21:34:30.009669+0000","last_unstale":"2024-09-19T21:35:34.151726+0000","last_undegraded":"2024-09-19T21:35:34.151726+0000","last_fullsized":"2024-09-19T21:35:34.151726+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:20:42.350074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":74326243,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":9677,"num_read_kb":34137,"num_write":275,"num_write_kb":1739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":61,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.023512+0000","last_change":"2024-09-19T21:34:22.954083+0000","last_active":"2024-09-19T21:35:36.023512+0000","last_peered":"2024-09-19T21:35:36.023512+0000","last_clean":"2024-09-19T21:35:36.023512+0000","last_became_active":"2024-09-19T21:34:20.952284+0000","last_became_peered":"2024-09-19T21:34:20.952284+0000","last_unstale":"2024-09-19T21:35:36.023512+0000","last_undegraded":"2024-09-19T21:35:36.023512+0000","last_fullsized":"2024-09-19T21:35:36.023512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:35.995140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":208,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.023198+0000","last_change":"2024-09-19T21:34:16.113091+0000","last_active":"2024-09-19T21:35:36.023198+0000","last_peered":"2024-09-19T21:35:36.023198+0000","last_clean":"2024-09-19T21:35:36.023198+0000","last_became_active":"2024-09-19T21:34:16.110915+0000","last_became_peered":"2024-09-19T21:34:16.110915+0000","last_unstale":"2024-09-19T21:35:36.023198+0000","last_undegraded":"2024-09-19T21:35:36.023198+0000","last_fullsized":"2024-09-19T21:35:36.023198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:52:40.830465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"65'196","reported_seq":5493,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.528959+0000","last_change":"2024-09-19T21:34:32.024996+0000","last_active":"2024-09-19T21:35:37.528959+0000","last_peered":"2024-09-19T21:35:37.528959+0000","last_clean":"2024-09-19T21:35:37.528959+0000","last_became_active":"2024-09-19T21:34:30.009636+0000","last_became_peered":"2024-09-19T21:34:30.009636+0000","last_unstale":"2024-09-19T21:35:37.528959+0000","last_undegraded":"2024-09-19T21:35:37.528959+0000","last_fullsized":"2024-09-19T21:35:37.528959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:11.666768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":71092768,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":5152,"num_read_kb":28698,"num_write":240,"num_write_kb":6684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"65'289","reported_seq":3469,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.068875+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:35:37.068875+0000","last_peered":"2024-09-19T21:35:37.068875+0000","last_clean":"2024-09-19T21:35:37.068875+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:35:37.068875+0000","last_undegraded":"2024-09-19T21:35:37.068875+0000","last_fullsized":"2024-09-19T21:35:37.068875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":53526751,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":3937,"num_read_kb":11248,"num_write":275,"num_write_kb":4362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"64'241","reported_seq":6812,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.075043+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:35:37.075043+0000","last_peered":"2024-09-19T21:35:37.075043+0000","last_clean":"2024-09-19T21:35:37.075043+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:35:37.075043+0000","last_undegraded":"2024-09-19T21:35:37.075043+0000","last_fullsized":"2024-09-19T21:35:37.075043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":26645082,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":7754,"num_read_kb":40241,"num_write":248,"num_write_kb":11625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"62'98","reported_seq":463,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T21:35:34.150184+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:35:34.150184+0000","last_peered":"2024-09-19T21:35:34.150184+0000","last_clean":"2024-09-19T21:35:34.150184+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:35:34.150184+0000","last_undegraded":"2024-09-19T21:35:34.150184+0000","last_fullsized":"2024-09-19T21:35:34.150184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":5714520,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":3,"num_read":418,"num_read_kb":5595,"num_write":104,"num_write_kb":2662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"65'96","reported_seq":642,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.893529+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:35:37.893529+0000","last_peered":"2024-09-19T21:35:37.893529+0000","last_clean":"2024-09-19T21:35:37.893529+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:35:37.893529+0000","last_undegraded":"2024-09-19T21:35:37.893529+0000","last_fullsized":"2024-09-19T21:35:37.893529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":3953152,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":2,"num_read":579,"num_read_kb":4662,"num_write":85,"num_write_kb":1892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"65'154","reported_seq":6318,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.187436+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:35:38.187436+0000","last_peered":"2024-09-19T21:35:38.187436+0000","last_clean":"2024-09-19T21:35:38.187436+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:35:38.187436+0000","last_undegraded":"2024-09-19T21:35:38.187436+0000","last_fullsized":"2024-09-19T21:35:38.187436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":50820140,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":6191,"num_read_kb":12926,"num_write":135,"num_write_kb":3195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"65'120","reported_seq":2360,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:38.173497+0000","last_change":"2024-09-19T21:34:32.024576+0000","last_active":"2024-09-19T21:35:38.173497+0000","last_peered":"2024-09-19T21:35:38.173497+0000","last_clean":"2024-09-19T21:35:38.173497+0000","last_became_active":"2024-09-19T21:34:30.006185+0000","last_became_peered":"2024-09-19T21:34:30.006185+0000","last_unstale":"2024-09-19T21:35:38.173497+0000","last_undegraded":"2024-09-19T21:35:38.173497+0000","last_fullsized":"2024-09-19T21:35:38.173497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:08:25.609727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":30462976,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":2248,"num_read_kb":43897,"num_write":133,"num_write_kb":4573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"65'499","reported_seq":3235,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.735490+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:35:37.735490+0000","last_peered":"2024-09-19T21:35:37.735490+0000","last_clean":"2024-09-19T21:35:37.735490+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:35:37.735490+0000","last_undegraded":"2024-09-19T21:35:37.735490+0000","last_fullsized":"2024-09-19T21:35:37.735490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":6135463,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":5463,"num_read_kb":7205,"num_write":595,"num_write_kb":1218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"64'306","reported_seq":6156,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-19T21:35:36.999497+0000","last_change":"2024-09-19T21:34:32.000772+0000","last_active":"2024-09-19T21:35:36.999497+0000","last_peered":"2024-09-19T21:35:36.999497+0000","last_clean":"2024-09-19T21:35:36.999497+0000","last_became_active":"2024-09-19T21:34:30.007945+0000","last_became_peered":"2024-09-19T21:34:30.007945+0000","last_unstale":"2024-09-19T21:35:36.999497+0000","last_undegraded":"2024-09-19T21:35:36.999497+0000","last_fullsized":"2024-09-19T21:35:36.999497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:20:07.932207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":41512960,"num_objects":48,"num_object_clones":16,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":7767,"num_read_kb":48090,"num_write":256,"num_write_kb":14094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"63'169","reported_seq":2175,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.112308+0000","last_change":"2024-09-19T21:34:32.023713+0000","last_active":"2024-09-19T21:35:37.112308+0000","last_peered":"2024-09-19T21:35:37.112308+0000","last_clean":"2024-09-19T21:35:37.112308+0000","last_became_active":"2024-09-19T21:34:30.003969+0000","last_became_peered":"2024-09-19T21:34:30.003969+0000","last_unstale":"2024-09-19T21:35:37.112308+0000","last_undegraded":"2024-09-19T21:35:37.112308+0000","last_fullsized":"2024-09-19T21:35:37.112308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:36.748615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":49490468,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":2692,"num_read_kb":10614,"num_write":129,"num_write_kb":1545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"65'141","reported_seq":4949,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.679074+0000","last_change":"2024-09-19T21:34:32.011754+0000","last_active":"2024-09-19T21:35:37.679074+0000","last_peered":"2024-09-19T21:35:37.679074+0000","last_clean":"2024-09-19T21:35:37.679074+0000","last_became_active":"2024-09-19T21:34:30.012126+0000","last_became_peered":"2024-09-19T21:34:30.012126+0000","last_unstale":"2024-09-19T21:35:37.679074+0000","last_undegraded":"2024-09-19T21:35:37.679074+0000","last_fullsized":"2024-09-19T21:35:37.679074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:05:32.355336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":57048064,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":3,"num_read":4840,"num_read_kb":57745,"num_write":148,"num_write_kb":5990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"65'179","reported_seq":6366,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T21:35:37.528744+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:35:37.528744+0000","last_peered":"2024-09-19T21:35:37.528744+0000","last_clean":"2024-09-19T21:35:37.528744+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:35:37.528744+0000","last_undegraded":"2024-09-19T21:35:37.528744+0000","last_fullsized":"2024-09-19T21:35:37.528744+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":34092672,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":11,"num_read":6360,"num_read_kb":17960,"num_write":205,"num_write_kb":4878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1177043917,"num_objects":1013,"num_object_clones":279,"num_object_copies":2026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1013,"num_whiteouts":276,"num_read":169962,"num_read_kb":705843,"num_write":8646,"num_write_kb":168216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":196811,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":259956736,"data_stored":337981338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7697,"ondisk_log_size":7697,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86744,"kb_used_data":51464,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94285096,"statfs":{"total":96636764160,"available":96547938304,"internally_reserved":0,"allocated":52699136,"data_stored":68852245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.59299999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Sep 19 21:35:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.626},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.61199999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66172,"kb_used_data":30960,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94305668,"statfs":{"total":96636764160,"available":96569004032,"internally_reserved":0,"allocated":31703040,"data_stored":35773246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79188,"kb_used_data":39876,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94292652,"statfs":{"total":96636764160,"available":96555675648,"internally_reserved":0,"allocated":40833024,"data_stored":47013753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.65000000000000002}]},{"osd":1,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99052,"kb_used_data":59744,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94272788,"statfs":{"total":96636764160,"available":96535334912,"internally_reserved":0,"allocated":61177856,"data_stored":86787318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.436}]},{"osd":2,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.621}]},{"osd":3,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.87},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.84799999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.98799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88644,"kb_used_data":45236,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94283196,"statfs":{"total":96636764160,"available":96545992704,"internally_reserved":0,"allocated":46321664,"data_stored":63817662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,0,61,81,103],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.54700000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.748},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.78200000000000003}]}]},{"osd":3,"up_from":12,"seq":51539607572,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71388,"kb_used_data":31868,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94300452,"statfs":{"total":96636764160,"available":96563662848,"internally_reserved":0,"allocated":32632832,"data_stored":38577840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.026},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.92200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45588480,"data_stored":63527528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59887616,"data_stored":85925267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40161280,"data_stored":46751286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31907840,"data_stored":38290639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31010816,"data_stored":35503592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":51400704,"data_stored":67983026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:35:39.913 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [5, 0] 2024-09-19T21:35:39.913 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.2', '5', '0'] 2024-09-19T21:35:39.913 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.2 5 0 2024-09-19T21:35:39.973 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:39.970+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:39.975 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:39.976 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:39.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:35:40.074 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:40.068+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:40.110 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:40.151 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:40.151 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:40.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:40.174 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:40.170+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:40.178 INFO:teuthology.orchestra.run.smithi122.stdout:3133 read 0xd35c00 thru 0xd439c4 (0xddc5 bytes) 2024-09-19T21:35:40.200 INFO:teuthology.orchestra.run.smithi122.stdout:3134 read 0xa57da00 thru 0xa5874bd (0x9abe bytes) 2024-09-19T21:35:40.203 INFO:teuthology.orchestra.run.smithi122.stdout:3138 read 0x609d400 thru 0x60a2b6b (0x576c bytes) 2024-09-19T21:35:40.208 INFO:teuthology.orchestra.run.smithi122.stdout:3140 read 0x2f29a00 thru 0x2f2e9fc (0x4ffd bytes) 2024-09-19T21:35:40.259 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:40.271 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:40.271 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:40.275 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:40.268+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:40.277 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:40.281 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:40.283 INFO:teuthology.orchestra.run.smithi122.stdout:3141 trunc from 0xba19800 to 0x92ae00 2024-09-19T21:35:40.302 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:40.335 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:40.354 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:40.376 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:40.374+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:40.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:40.378 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:40.468 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:40.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:40.468 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:40.468 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:40.476 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:40.472+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:40.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:40.577 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:40.574+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:40.584 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:40.584 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:40.584 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:40.584 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:40.584 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:40.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:40.679 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:40.674+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 44.974764487999998, 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:40.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17094 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3502080~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375927+0000", 2024-09-19T21:35:40.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34745925999999999, 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065023733, 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17094 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375927+0000", 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375927+0000", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.372015+0000", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9960885 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375932+0000", 2024-09-19T21:35:40.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0039167259999999997 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375942+0000", 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7070000000000004e-06 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375950+0000", 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9189999999999999e-06 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.440675+0000", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064725014999999997 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.440749+0000", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3827999999999999e-05 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.440951+0000", 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00020211400000000001 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34742509199999999, 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:35:40.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:35:40.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34737262600000002, 2024-09-19T21:35:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:35:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34734207900000003, 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.747 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34731948200000001, 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:35:40.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.755 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34729703000000001, 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:35:40.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:35:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34728273399999998, 2024-09-19T21:35:40.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:35:40.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34726610699999999, 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.765 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:40.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:35:40.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34724747099999997, 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:35:40.773 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34722080599999999, 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:35:40.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34721059500000001, 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:35:40.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:35:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34719810899999998, 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.783 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:40.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.787 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34332351900000002, 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:35:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.343306, 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.343268446, 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34325060299999999, 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:35:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34323815400000002, 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:35:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.34322746199999998, 2024-09-19T21:35:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:35:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.343215885, 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:35:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:40.819 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:40.821 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:40.816+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:40.821 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:40.921 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:40.916+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:40.949 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:40.950 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:40.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:41.023 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:41.016+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:41.039 INFO:teuthology.orchestra.run.smithi122.stdout:3142 punch from 0x6c4c00 to 0x6c6200, (0x1600 bytes) 2024-09-19T21:35:41.041 INFO:teuthology.orchestra.run.smithi122.stdout:3143 punch from 0x6ea200 to 0x6f5000, (0xae00 bytes) 2024-09-19T21:35:41.070 INFO:teuthology.orchestra.run.smithi012.stderr:set 2.2 pg_upmap mapping to [5,0] 2024-09-19T21:35:41.080 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.080 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:41.080 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:41.080 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:41.080 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.086 INFO:teuthology.orchestra.run.smithi122.stdout:3144 write 0xc9a6e00 thru 0xc9ac3cb (0x55cc bytes) 2024-09-19T21:35:41.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:41.099 INFO:teuthology.orchestra.run.smithi122.stdout:3145 trunc from 0xc9ac400 to 0x6fe0600 2024-09-19T21:35:41.123 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:41.122+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:41.197 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.197 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:41.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8372426369999992, 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.198 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:41.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:41.202 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8372303290000005, 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.206 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8044990900000002, 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:41.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:41.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:41.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5747413970000004, 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:41.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:41.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5747179619999994, 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:41.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.220 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5746962339999993, 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:41.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:41.225 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5746755019999998, 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:41.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5746410659999999, 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:41.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:41.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5745982689999991, 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:41.238 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5693727590000002, 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:41.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5693504609999991, 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:41.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:41.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:41.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5693196060000005, 2024-09-19T21:35:41.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:41.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:41.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:41.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5692768969999999, 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.253 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5692529099999994, 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:41.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5692215059999999, 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:41.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5691963399999995, 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.270 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5691512620000001, 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:41.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5691245180000006, 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:41.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5691112260000004, 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:41.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:41.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.5690943070000003, 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:41.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:41.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.289 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.290 INFO:teuthology.orchestra.run.smithi122.stdout:3147 punch from 0x30fe00 to 0x31f400, (0xf600 bytes) 2024-09-19T21:35:41.290 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:41.284+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:41.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:41.392 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:41.384+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:41.455 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.456 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:41.456 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:41.456 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:41.491 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:41.484+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:41.569 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:41.624 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:41.622+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 85.756191758, 2024-09-19T21:35:41.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:41.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:41.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4366244269999999, 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4366126609999998, 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4365997680000007, 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:35:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4365877450000006, 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:35:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:35:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4365701009999992, 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:35:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:35:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4365543049999996, 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:35:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4365424539999996, 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:35:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4364736100000002, 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:35:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:35:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4364454700000007, 2024-09-19T21:35:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:35:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:35:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4364223410000001, 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:35:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4216457210000009, 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4143094059999992, 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:35:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:35:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4142885159999992, 2024-09-19T21:35:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:35:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4072129760000003, 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.4071924320000004, 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.2268383610000004, 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:35:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.2268279339999992, 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:35:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.2268175049999996, 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:35:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:35:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.2268074420000001, 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:35:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:41.788 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.789 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:41.690+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:41.790 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:41.836 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:41.834+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:41.894 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:41.888+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:41.907 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:41.908 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:41.908 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:41.908 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:41.916 INFO:teuthology.orchestra.run.smithi122.stdout:3148 trunc from 0x6fe0600 to 0x65f3800 2024-09-19T21:35:41.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:41.995 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:41.994+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:42.036 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:42.036 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:42.036 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:42.036 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:42.036 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:42.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:42.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:35:42.096 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:42.092+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:42.185 INFO:teuthology.orchestra.run.smithi122.stdout:3150 trunc from 0x65f3800 to 0x1b96000 2024-09-19T21:35:42.197 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:42.192+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4554.0:3002 3.1d 3:bcebfb12:::rbd_data.11064af7243d.0000000000000001:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.008605539000001, 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.033057975000000003, 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4554", 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2452545263", 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3002 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227441+0000", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227439+0000", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227450+0000", 2024-09-19T21:35:42.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0699e-05 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227477+0000", 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6818e-05 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227491+0000", 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4474000000000001e-05 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227619+0000", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012767400000000001 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.227914+0000", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029488000000000002 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:41.260499+0000", 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.032585451000000001 2024-09-19T21:35:42.243 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1471 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534577730999999, 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029212798000000002, 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1471 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701469+0000", 2024-09-19T21:35:42.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701468+0000", 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701472+0000", 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4879999999999999e-06 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701484+0000", 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1843e-05 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701495+0000", 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1449e-05 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730532+0000", 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029036600999999999 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730599+0000", 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7460999999999997e-05 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730682+0000", 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2548999999999999e-05 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.248 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1473 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534547904, 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029383548999999998, 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1473 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701499+0000", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701498+0000", 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701500+0000", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701504+0000", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3189999999999999e-06 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701506+0000", 2024-09-19T21:35:42.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730718+0000", 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029211509 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730793+0000", 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4633000000000002e-05 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730882+0000", 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9722000000000006e-05 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.253 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1474 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534538218000002, 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02955354, 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1474 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701508+0000", 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701509+0000", 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.268e-06 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701512+0000", 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7609999999999999e-06 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701515+0000", 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5050000000000002e-06 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730921+0000", 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029406163999999999 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.730976+0000", 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5448000000000002e-05 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731062+0000", 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5611000000000004e-05 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1477 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534529892000002, 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029774724999999998, 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1477 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.259 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701517+0000", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701518+0000", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.153e-06 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701521+0000", 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:42.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701523+0000", 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3910000000000001e-06 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731102+0000", 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029578912999999998 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731169+0000", 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7519000000000002e-05 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731291+0000", 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000122118 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1484 3.e 3:774c54a4:::rbd_data.110682966ae8.0000000000000001:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534519500000002, 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029992062999999999, 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1484 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701527+0000", 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:42.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701528+0000", 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1689999999999999e-06 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701531+0000", 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1259999999999998e-06 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701535+0000", 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5030000000000002e-06 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731323+0000", 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029788445 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731399+0000", 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5767999999999997e-05 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731519+0000", 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012027999999999999 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1487 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.534509004, 2024-09-19T21:35:42.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030205308, 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1487 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701538+0000", 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701537+0000", 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701539+0000", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.252e-06 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701542+0000", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9330000000000001e-06 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.701545+0000", 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.117e-06 2024-09-19T21:35:42.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731559+0000", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030014418000000001 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731621+0000", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2201999999999998e-05 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.731743+0000", 2024-09-19T21:35:42.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121624 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1548 3.e 3:716e054e:::rbd_data.110682966ae8.0000000000000004:a [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.532064916000003, 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277033000000001, 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1548 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703982+0000", 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703981+0000", 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703984+0000", 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9299999999999999e-06 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.703991+0000", 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8619999999999997e-06 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.704003+0000", 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1742999999999999e-05 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733122+0000", 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029119042000000001 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733185+0000", 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3088000000000003e-05 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.733259+0000", 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3817000000000002e-05 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.506847530000002, 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15066 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6164480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611252838999999, 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029164998000000001, 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15066 2024-09-19T21:35:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624794+0000", 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624793+0000", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624797+0000", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4580000000000002e-06 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624803+0000", 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6800000000000004e-06 2024-09-19T21:35:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624809+0000", 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8969999999999996e-06 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653805+0000", 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.028996141999999999 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653889+0000", 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3906000000000006e-05 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653959+0000", 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9604000000000006e-05 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15067 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6178816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611224230000001, 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029307910999999999, 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15067 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624822+0000", 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624831+0000", 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2019999999999993e-06 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624840+0000", 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5639999999999997e-06 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624846+0000", 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4740000000000002e-06 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.653998+0000", 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029151795000000001 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654062+0000", 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4289999999999996e-05 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654130+0000", 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8034000000000006e-05 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15068 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6193152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611196229000001, 2024-09-19T21:35:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029394144000000001, 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15068 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624850+0000", 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624853+0000", 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8339999999999999e-06 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624860+0000", 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7689999999999999e-06 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624866+0000", 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0639999999999999e-06 2024-09-19T21:35:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654143+0000", 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029277609999999999 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654189+0000", 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5892e-05 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654245+0000", 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5383000000000002e-05 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15069 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6207488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611175883000001, 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029542570000000001, 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15069 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624871+0000", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624870+0000", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624873+0000", 2024-09-19T21:35:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-06 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624880+0000", 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624886+0000", 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654272+0000", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029385674000000001 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654328+0000", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6156999999999998e-05 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654413+0000", 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999998e-05 2024-09-19T21:35:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15070 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6221824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611155695000001, 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.02965835, 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15070 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624891+0000", 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624894+0000", 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9900000000000002e-06 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624903+0000", 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1220000000000001e-06 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624917+0000", 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3973e-05 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654426+0000", 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029508869 2024-09-19T21:35:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654475+0000", 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9274999999999998e-05 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654549+0000", 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4492000000000004e-05 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15071 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6236160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611123662000001, 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029922358, 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15071 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624923+0000", 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624922+0000", 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624926+0000", 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2650000000000001e-06 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624933+0000", 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4760000000000001e-06 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624939+0000", 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9819999999999996e-06 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654596+0000", 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029657246000000002 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654708+0000", 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112194 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654845+0000", 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013685999999999999 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15072 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6250496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611102249999998, 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030057148999999998, 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15072 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624944+0000", 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624947+0000", 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.326e-06 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624954+0000", 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6590000000000001e-06 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624959+0000", 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3630000000000002e-06 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654863+0000", 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029903855999999999 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.654917+0000", 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4487000000000002e-05 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655002+0000", 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4300000000000003e-05 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15073 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6264832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611081247000001, 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030212118999999999, 2024-09-19T21:35:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15073 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624965+0000", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624964+0000", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624968+0000", 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.16e-06 2024-09-19T21:35:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624975+0000", 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9600000000000003e-06 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624981+0000", 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7980000000000002e-06 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655037+0000", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030056058 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655093+0000", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6223e-05 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655177+0000", 2024-09-19T21:35:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4654000000000006e-05 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:15074 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 6279168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.611061022000001, 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030446608, 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 15074 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624986+0000", 2024-09-19T21:35:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624985+0000", 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.624988+0000", 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1370000000000002e-06 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625010+0000", 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1778000000000001e-05 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.625017+0000", 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5259999999999999e-06 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655204+0000", 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.030186983000000001 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655303+0000", 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9926999999999994e-05 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:14.655432+0000", 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012875899999999999 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18639 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 731136~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.214083551000002, 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029303412000000001, 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18639 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021963+0000", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967296 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021964+0000", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021967+0000", 2024-09-19T21:35:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021970+0000", 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2349999999999999e-06 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051070+0000", 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029100002 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051159+0000", 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9033e-05 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051266+0000", 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107069 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5043.0:18640 3.16 3:6d89d3d7:::rbd_data.11066d5522c8.0000000000000002:12 [read 745472~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.214063181, 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029524404000000001, 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5043", 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18640 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021983+0000", 2024-09-19T21:35:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021985+0000", 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9750000000000001e-06 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021988+0000", 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9720000000000001e-06 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.021991+0000", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6369999999999999e-06 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051317+0000", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029326609 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051398+0000", 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0705999999999996e-05 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:15.051508+0000", 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109948 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:42.336 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:42.338 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:42.332+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:42.338 INFO:teuthology.orchestra.run.smithi122.stdout:3159 write 0xdcdcc00 thru 0xdce2858 (0x5c59 bytes) 2024-09-19T21:35:42.339 INFO:teuthology.orchestra.run.smithi122.stdout:3160 read 0x8d79e00 thru 0x8d89189 (0xf38a bytes) 2024-09-19T21:35:42.339 INFO:teuthology.orchestra.run.smithi122.stdout:3161 trunc from 0xdce2a00 to 0x70c6c00 2024-09-19T21:35:42.339 INFO:teuthology.orchestra.run.smithi122.stdout:3163 read 0x5efbc00 thru 0x5f08322 (0xc723 bytes) 2024-09-19T21:35:42.339 INFO:teuthology.orchestra.run.smithi122.stdout:3164 write 0x3973e00 thru 0x3978272 (0x4473 bytes) 2024-09-19T21:35:42.339 INFO:teuthology.orchestra.run.smithi122.stdout:3167 trunc from 0x70c6c00 to 0x9698a00 2024-09-19T21:35:42.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:42.377 INFO:teuthology.orchestra.run.smithi122.stdout:3169 punch from 0xdbe000 to 0xdcc600, (0xe600 bytes) 2024-09-19T21:35:42.380 INFO:teuthology.orchestra.run.smithi122.stdout:3171 read 0x7b01800 thru 0x7b07b44 (0x6345 bytes) 2024-09-19T21:35:42.380 INFO:teuthology.orchestra.run.smithi122.stdout:3173 write 0x8545000 thru 0x854e2fc (0x92fd bytes) 2024-09-19T21:35:42.380 INFO:teuthology.orchestra.run.smithi122.stdout:3175 punch from 0x8488c00 to 0x8493a00, (0xae00 bytes) 2024-09-19T21:35:42.383 INFO:teuthology.orchestra.run.smithi122.stdout:3177 punch from 0x20ada00 to 0x20b6000, (0x8600 bytes) 2024-09-19T21:35:42.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:42.391 INFO:teuthology.orchestra.run.smithi122.stdout:3179 trunc from 0x9698a00 to 0xc03f000 2024-09-19T21:35:42.438 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:42.432+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:42.466 INFO:teuthology.orchestra.run.smithi122.stdout:3181 read 0x4ae9400 thru 0x4aee722 (0x5323 bytes) 2024-09-19T21:35:42.467 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:42.467 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:42.467 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:42.467 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:42.469 INFO:teuthology.orchestra.run.smithi122.stdout:3182 write 0x55ef600 thru 0x55fd244 (0xdc45 bytes) 2024-09-19T21:35:42.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:42.526 INFO:teuthology.orchestra.run.smithi122.stdout:3183 write 0xec7a000 thru 0xec87cb3 (0xdcb4 bytes) 2024-09-19T21:35:42.528 INFO:teuthology.orchestra.run.smithi122.stdout:3184 write 0xae01a00 thru 0xae0a283 (0x8884 bytes) 2024-09-19T21:35:42.528 INFO:teuthology.orchestra.run.smithi122.stdout:3185 read 0xca22e00 thru 0xca29fe7 (0x71e8 bytes) 2024-09-19T21:35:42.528 INFO:teuthology.orchestra.run.smithi122.stdout:3187 punch from 0x7331400 to 0x733fc00, (0xe800 bytes) 2024-09-19T21:35:42.531 INFO:teuthology.orchestra.run.smithi122.stdout:3188 read 0x659000 thru 0x65f16e (0x616f bytes) 2024-09-19T21:35:42.538 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:42.539 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:42.534+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:42.542 INFO:teuthology.orchestra.run.smithi122.stdout:3189 write 0x64b7000 thru 0x64be2e5 (0x72e6 bytes) 2024-09-19T21:35:42.543 INFO:teuthology.orchestra.run.smithi122.stdout:3192 read 0xd45d400 thru 0xd466de4 (0x99e5 bytes) 2024-09-19T21:35:42.543 INFO:teuthology.orchestra.run.smithi122.stdout:3195 read 0x95cbe00 thru 0x95d2172 (0x6373 bytes) 2024-09-19T21:35:42.543 INFO:teuthology.orchestra.run.smithi122.stdout:3197 clone 22 order 21 su 65536 sc 12 2024-09-19T21:35:42.567 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:42.599 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:42.599 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:42.599 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:42.600 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:42.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:42.622 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:42.641 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:42.657 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:42.652+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:42.662 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:42.665 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:42.665 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:42.714 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:42.714 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:42.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:42.741 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:42.738+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:42.830 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:42.830 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:42.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:42.842 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:42.838+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:42.943 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:42.938+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:42.950 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:42.950 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:42.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:43.044 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:43.036+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:43.068 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.068 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:35:43.145 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:43.140+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:43.185 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.186 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:43.229 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone21 from 0xec87e00 (overlap 0x92ae00) to 0x3aac00 2024-09-19T21:35:43.246 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:43.242+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:43.319 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.319 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:43.346 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:43.340+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:43.447 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:43.440+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:43.467 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.467 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.478 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:43.548 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:43.540+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:43.599 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.599 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.609 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:43.649 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:43.644+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:43.713 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.714 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:43.750 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:43.744+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:43.835 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.835 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:43.851 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:43.844+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:43.952 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:43.950+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:43.954 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:43.954 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:43.964 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:44.053 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:44.048+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:44.083 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.083 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:44.154 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:44.148+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:44.200 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.200 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:44.256 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:44.252+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:44.329 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.329 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:44.384 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:44.382+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:44.457 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.457 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.458 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:44.454+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:44.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:44.558 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:44.554+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:44.568 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.568 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.575 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-19T21:35:44.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:44.659 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:44.652+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:44.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:44.704 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.704 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:44.760 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:44.758+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:44.826 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:44.830 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:44.830 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:44.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:44.848 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:44.860 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:44.858+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:44.867 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:44.886 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:44.911 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:44.932 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:44.935 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.959 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702721716999999, 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702708178000002, 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702682438, 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702668224, 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:44.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702647306999999, 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:44.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702631930999999, 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:44.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702617003, 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:44.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702564382999999, 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:44.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:44.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:44.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702514805, 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:44.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:45.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:45.003 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.70249759, 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:45.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702478452000001, 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:45.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702463603999998, 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:45.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.702440611, 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:45.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.701708319000002, 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:45.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:45.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.701658008999999, 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:45.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.701595696999998, 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.700265064, 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:45.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:45.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.700129486000002, 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:45.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.700052726999999, 2024-09-19T21:35:45.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:45.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 30.699994003, 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:45.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:45.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.054 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:45.056 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:45.054+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:45.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:35:45.156 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:45.154+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:45.172 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.173 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:45.173 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:45.173 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:45.258 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:45.252+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:45.292 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.292 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:45.292 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:45.292 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:45.293 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:45.309 INFO:teuthology.orchestra.run.smithi122.stdout:3199 trunc from 0xec87e00 to 0xc6ee400 2024-09-19T21:35:45.359 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:45.354+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:45.414 INFO:teuthology.orchestra.run.smithi122.stdout:3203 trunc from 0xc6ee400 to 0x60d8a00 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.214731143000002, 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:35:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:35:45.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.214151231999999, 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:35:45.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.214088584999999, 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:35:45.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:35:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.214022265000001, 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:35:45.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:35:45.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.434 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213318587, 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:35:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213245765, 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:35:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:35:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213232027, 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.444 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:45.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:35:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213214338, 2024-09-19T21:35:45.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:35:45.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213163951999999, 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:35:45.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213137063000001, 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:35:45.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213096487000001, 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:35:45.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213074873, 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:35:45.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.473 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.213055674, 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:35:45.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7243381800000002, 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:35:45.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.483 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7240709029999999, 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:35:45.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:35:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7237380719999997, 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:35:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7231909280000002, 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:35:45.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:35:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:35:45.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7231145899999998, 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.7227823139999998, 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:35:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:35:45.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.722479388, 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:35:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:35:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.511 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.511 INFO:teuthology.orchestra.run.smithi122.stdout:3209 clone 23 order 18 su 65536 sc 4 2024-09-19T21:35:45.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:45.513 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:45.510+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:45.613 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:45.610+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:45.624 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.624 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:45.624 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:45.624 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:45.714 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:45.710+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:45.743 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:45.744 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:45.815 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:45.808+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:45.867 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:45.867 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 50.114932826999997, 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:45.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:45.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:45.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4875934309999996, 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.875 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.879 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.487540965, 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4875104180000003, 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:35:45.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:45.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:35:45.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4874878210000002, 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:35:45.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4874653689999997, 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:35:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4874510729999999, 2024-09-19T21:35:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:35:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.487434446, 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:35:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:35:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4874158099999999, 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:35:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:35:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.916 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4873891449999999, 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:35:45.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.921 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4873789339999997, 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:35:45.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4873664480000004, 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:45.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:35:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4834918579999998, 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:35:45.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4834743389999998, 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:35:45.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:35:45.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4834367850000003, 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:35:45.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:35:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4834189420000001, 2024-09-19T21:35:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:45.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4834064930000004, 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:35:45.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4833958010000003, 2024-09-19T21:35:45.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:35:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.4833842239999999, 2024-09-19T21:35:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:45.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.067412268, 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:35:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:45.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:45.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:45.973 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:45.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:45.975 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:45.970+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:46.075 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:46.070+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:46.084 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:46.084 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:46.084 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T21:35:46.084 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:35:46.085 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:35:46.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T21:35:46.092 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.092 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:46.092 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:46.093 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.102 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:46.115 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:35:46.175 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:46.174+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:46.203 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.203 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "" 2024-09-19T21:35:46.204 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.209 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.210 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:46.210 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:46.210 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:46.210 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:46.271 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone22 2024-09-19T21:35:46.277 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:46.274+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:46.364 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.364 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:46.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.003427595, 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:46.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:46.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.369 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.370 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:46.370 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.003415286999999, 2024-09-19T21:35:46.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:46.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:46.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:46.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.970684048000001, 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.381 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.740926354999999, 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:46.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.74090292, 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:46.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:46.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:46.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:46.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:46.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.740881192, 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:46.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:46.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.74086046, 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:46.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.740826024, 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:46.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:46.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:46.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.740783227, 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:46.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:46.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:46.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735557717000001, 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.415 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:46.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.419 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735535419, 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:46.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:46.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735504563999999, 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:46.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735461855, 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:46.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.433 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735437868, 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.434 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735406464, 2024-09-19T21:35:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:46.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735381298, 2024-09-19T21:35:46.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:46.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:46.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735336220000001, 2024-09-19T21:35:46.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:46.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:46.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:46.450 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735309475999999, 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.454 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735296183999999, 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:46.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:46.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.735279264999999, 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:46.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:46.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.463 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:46.465 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:46.462+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:46.565 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:46.562+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:46.627 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.627 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:46.627 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:46.627 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:46.666 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:46.660+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:46.750 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.750 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:46.750 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:46.750 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:46.750 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:46.767 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:46.762+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:46.868 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:46.866+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:46.883 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:46.883 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:46.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.947500124000001, 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:46.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:46.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:46.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627932792999999, 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.891 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:46.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627921026999999, 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:35:46.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627908134, 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627896111, 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.904 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:46.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627878467, 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.909 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:35:46.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627862671000001, 2024-09-19T21:35:46.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:35:46.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:35:46.917 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627850820000001, 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.918 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:35:46.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.922 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627781976, 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:46.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:35:46.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627753836, 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:35:46.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.627730707, 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.612954087, 2024-09-19T21:35:46.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:35:46.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:35:46.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.605617772, 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:35:46.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.605596882, 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:35:46.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:35:46.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.598521342, 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:35:46.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:46.953 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.598500798, 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:46.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:35:46.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.418146727, 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:35:46.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.4181363, 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:46.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:35:46.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.418125871000001, 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.418115808, 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:46.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:35:46.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.976 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:46.977 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:46.977 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:46.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:46.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:46.979 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:46.974+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:47.079 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:47.078+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:47.116 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5102.0:320 3.11 3:8d054e1a:::rbd_data.11064e5a13b9.000000000000003b:1a [read 0~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:47.150202+0000", 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.000332004, 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00036697699999999999, 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5102", 2024-09-19T21:35:47.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2731104914", 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 320 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150202+0000", 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150202+0000", 2024-09-19T21:35:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150201+0000", 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150208+0000", 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2480000000000001e-06 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:47.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150229+0000", 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1691999999999998e-05 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150240+0000", 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1165000000000001e-05 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150320+0000", 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9985000000000001e-05 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150423+0000", 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102995 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150571+0000", 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000147436 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5102.0:323 3.11 3:8d054e1a:::rbd_data.11064e5a13b9.000000000000003b:1a [read 36864~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:47.150254+0000", 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00028058, 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000426203, 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5102", 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2731104914", 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 323 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:47.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150254+0000", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150254+0000", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150253+0000", 2024-09-19T21:35:47.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150257+0000", 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9910000000000004e-06 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150269+0000", 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1501999999999999e-05 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150275+0000", 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6309999999999997e-06 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:47.150608+0000", 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033280700000000001 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:47.160 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:47.161 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 2 2024-09-19T21:35:47.161 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:47.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:47.172 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:47.179 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:47.178+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:47.191 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:47.210 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:47.234 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:47.262 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.262 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.263 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:47.272 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:47.287 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:47.287 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:47.286+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:47.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:47.290 INFO:teuthology.orchestra.run.smithi122.stdout:3212 trunc from 0x60d8a00 to 0x15ebc00 2024-09-19T21:35:47.329 INFO:teuthology.orchestra.run.smithi122.stdout:3213 read 0x666a00 thru 0x668b7f (0x2180 bytes) 2024-09-19T21:35:47.361 INFO:teuthology.orchestra.run.smithi122.stdout:3214 write 0x3005800 thru 0x3012821 (0xd022 bytes) 2024-09-19T21:35:47.363 INFO:teuthology.orchestra.run.smithi122.stdout:3215 punch from 0xa7da00 to 0xa85600, (0x7c00 bytes) 2024-09-19T21:35:47.364 INFO:teuthology.orchestra.run.smithi122.stdout:3217 read 0x2840600 thru 0x28456a9 (0x50aa bytes) 2024-09-19T21:35:47.374 INFO:teuthology.orchestra.run.smithi122.stdout:3219 trunc from 0x3012a00 to 0x630fa00 2024-09-19T21:35:47.381 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:47.378+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:47.391 INFO:teuthology.orchestra.run.smithi122.stdout:3222 read 0x5e8d800 thru 0x5e9967c (0xbe7d bytes) 2024-09-19T21:35:47.392 INFO:teuthology.orchestra.run.smithi122.stdout:3225 read 0x20c4e00 thru 0x20cda6c (0x8c6d bytes) 2024-09-19T21:35:47.392 INFO:teuthology.orchestra.run.smithi122.stdout:3229 punch from 0x5b43a00 to 0x5b44600, (0xc00 bytes) 2024-09-19T21:35:47.401 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.402 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:47.468 INFO:teuthology.orchestra.run.smithi122.stdout:3230 write 0xded4a00 thru 0xded6fa9 (0x25aa bytes) 2024-09-19T21:35:47.470 INFO:teuthology.orchestra.run.smithi122.stdout:3233 read 0xaf75400 thru 0xaf808b6 (0xb4b7 bytes) 2024-09-19T21:35:47.470 INFO:teuthology.orchestra.run.smithi122.stdout:3234 punch from 0xbb20000 to 0xbb27200, (0x7200 bytes) 2024-09-19T21:35:47.481 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:47.478+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:47.484 INFO:teuthology.orchestra.run.smithi122.stdout:3235 trunc from 0xded7000 to 0x6b6aa00 2024-09-19T21:35:47.518 INFO:teuthology.orchestra.run.smithi122.stdout:3237 read 0x582e800 thru 0x5830b4a (0x234b bytes) 2024-09-19T21:35:47.521 INFO:teuthology.orchestra.run.smithi122.stdout:3239 write 0x1581600 thru 0x158d13c (0xbb3d bytes) 2024-09-19T21:35:47.522 INFO:teuthology.orchestra.run.smithi122.stdout:3241 read 0x4cf6c00 thru 0x4cff10e (0x850f bytes) 2024-09-19T21:35:47.522 INFO:teuthology.orchestra.run.smithi122.stdout:3242 punch from 0x3cd4600 to 0x3ce4200, (0xfc00 bytes) 2024-09-19T21:35:47.524 INFO:teuthology.orchestra.run.smithi122.stdout:3244 read 0x3eb2a00 thru 0x3eb31ec (0x7ed bytes) 2024-09-19T21:35:47.524 INFO:teuthology.orchestra.run.smithi122.stdout:3246 read 0x2669c00 thru 0x266ca04 (0x2e05 bytes) 2024-09-19T21:35:47.524 INFO:teuthology.orchestra.run.smithi122.stdout:3248 punch from 0x1f0bc00 to 0x1f15600, (0x9a00 bytes) 2024-09-19T21:35:47.532 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.532 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.536 INFO:teuthology.orchestra.run.smithi122.stdout:3253 trunc from 0x6b6aa00 to 0xc484600 2024-09-19T21:35:47.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:47.557 INFO:teuthology.orchestra.run.smithi122.stdout:3254 write 0x5792a00 thru 0x57a1370 (0xe971 bytes) 2024-09-19T21:35:47.568 INFO:teuthology.orchestra.run.smithi122.stdout:3255 trunc from 0xc484600 to 0xe6d0400 2024-09-19T21:35:47.582 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:47.578+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:47.609 INFO:teuthology.orchestra.run.smithi122.stdout:3259 trunc from 0xe6d0400 to 0xa1ea200 2024-09-19T21:35:47.646 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.646 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:47.662 INFO:teuthology.orchestra.run.smithi122.stdout:3260 write 0xbad9000 thru 0xbadb43f (0x2440 bytes) 2024-09-19T21:35:47.683 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:47.676+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:47.684 INFO:teuthology.orchestra.run.smithi122.stdout:3261 trunc from 0xbadb600 to 0x2588c00 2024-09-19T21:35:47.766 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.766 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.771 INFO:teuthology.orchestra.run.smithi122.stdout:3262 write 0xc92d600 thru 0xc9379d8 (0xa3d9 bytes) 2024-09-19T21:35:47.772 INFO:teuthology.orchestra.run.smithi122.stdout:3264 read 0x6264200 thru 0x6271306 (0xd107 bytes) 2024-09-19T21:35:47.773 INFO:teuthology.orchestra.run.smithi122.stdout:3265 read 0x6dd9e00 thru 0x6de0c41 (0x6e42 bytes) 2024-09-19T21:35:47.773 INFO:teuthology.orchestra.run.smithi122.stdout:3267 read 0xc208800 thru 0xc20c5b5 (0x3db6 bytes) 2024-09-19T21:35:47.773 INFO:teuthology.orchestra.run.smithi122.stdout:3268 read 0xb819200 thru 0xb819d26 (0xb27 bytes) 2024-09-19T21:35:47.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:47.784 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:47.782+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:47.805 INFO:teuthology.orchestra.run.smithi122.stdout:3269 trunc from 0xc937a00 to 0x518c00 2024-09-19T21:35:47.869 INFO:teuthology.orchestra.run.smithi122.stdout:3273 trunc from 0x518c00 to 0x2947000 2024-09-19T21:35:47.885 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:47.880+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:47.921 INFO:teuthology.orchestra.run.smithi122.stdout:3274 write 0x4e10200 thru 0x4e1f928 (0xf729 bytes) 2024-09-19T21:35:47.936 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:47.936 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:47.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:47.977 INFO:teuthology.orchestra.run.smithi122.stdout:3275 write 0xde09800 thru 0xde14507 (0xad08 bytes) 2024-09-19T21:35:47.982 INFO:teuthology.orchestra.run.smithi122.stdout:3276 trunc from 0xde14600 to 0xcfbb600 2024-09-19T21:35:47.985 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:47.980+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:48.033 INFO:teuthology.orchestra.run.smithi122.stdout:3278 read 0x31c3800 thru 0x31d11dc (0xd9dd bytes) 2024-09-19T21:35:48.051 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.051 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.057 INFO:teuthology.orchestra.run.smithi122.stdout:3281 trunc from 0xcfbb600 to 0x3e27c00 2024-09-19T21:35:48.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:48.105 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:48.100+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:48.105 INFO:teuthology.orchestra.run.smithi122.stdout:3282 read 0xb4ae00 thru 0xb53b2e (0x8d2f bytes) 2024-09-19T21:35:48.108 INFO:teuthology.orchestra.run.smithi122.stdout:3283 clone 24 order 24 su 2097152 sc 11 2024-09-19T21:35:48.132 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:35:48.187 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:48.180+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:48.221 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.222 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:48.273 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone23 from 0x3e27c00 (overlap 0x518c00) to 0x28c600 2024-09-19T21:35:48.288 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:48.286+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:48.336 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.336 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:48.390 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:48.382+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.451 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:48.490 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:48.482+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:48.567 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.567 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:48.592 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:48.584+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:48.691 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:48.684+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:48.700 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.700 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:48.792 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:48.784+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:48.811 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.811 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.821 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:48.885 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-19T21:35:48.893 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:48.886+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:48.930 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:48.930 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:48.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:48.993 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:48.986+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:49.065 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:49.065 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:49.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:49.093 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:49.088+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:49.147 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:35:49.195 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:49.188+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:49.234 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:49.235 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:49.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:49.288 INFO:teuthology.orchestra.run.smithi122.stdout:3284 read 0x279f800 thru 0x27a8d27 (0x9528 bytes) 2024-09-19T21:35:49.296 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:49.290+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:49.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:49.310 INFO:teuthology.orchestra.run.smithi122.stdout:3285 trunc from 0x3e27c00 to 0x30800 2024-09-19T21:35:49.358 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:49.358 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:49.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:49.397 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:49.390+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:49.417 INFO:teuthology.orchestra.run.smithi122.stdout:3287 write 0x7e8aa00 thru 0x7e901cc (0x57cd bytes) 2024-09-19T21:35:49.420 INFO:teuthology.orchestra.run.smithi122.stdout:3288 read 0x5352e00 thru 0x53576e1 (0x48e2 bytes) 2024-09-19T21:35:49.420 INFO:teuthology.orchestra.run.smithi122.stdout:3290 read 0x289c200 thru 0x28a4476 (0x8277 bytes) 2024-09-19T21:35:49.420 INFO:teuthology.orchestra.run.smithi122.stdout:3291 punch from 0x32f1000 to 0x32fa400, (0x9400 bytes) 2024-09-19T21:35:49.421 INFO:teuthology.orchestra.run.smithi122.stdout:3295 read 0x40a1400 thru 0x40aa0f0 (0x8cf1 bytes) 2024-09-19T21:35:49.421 INFO:teuthology.orchestra.run.smithi122.stdout:3296 punch from 0x2b96e00 to 0x2ba0a00, (0x9c00 bytes) 2024-09-19T21:35:49.447 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:49.462 INFO:teuthology.orchestra.run.smithi122.stdout:3298 write 0xb517a00 thru 0xb51f880 (0x7e81 bytes) 2024-09-19T21:35:49.464 INFO:teuthology.orchestra.run.smithi122.stdout:3299 punch from 0x751200 to 0x75d800, (0xc600 bytes) 2024-09-19T21:35:49.474 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:49.483 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:49.483 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:49.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:49.493 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:49.497 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:49.490+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:49.508 INFO:teuthology.orchestra.run.smithi122.stdout:3301 write 0xe526200 thru 0xe529f33 (0x3d34 bytes) 2024-09-19T21:35:49.509 INFO:teuthology.orchestra.run.smithi122.stdout:3302 read 0xc6e1600 thru 0xc6ea932 (0x9333 bytes) 2024-09-19T21:35:49.509 INFO:teuthology.orchestra.run.smithi122.stdout:3306 write 0xa5cca00 thru 0xa5d4670 (0x7c71 bytes) 2024-09-19T21:35:49.510 INFO:teuthology.orchestra.run.smithi122.stdout:3307 read 0xc0d0a00 thru 0xc0d6692 (0x5c93 bytes) 2024-09-19T21:35:49.510 INFO:teuthology.orchestra.run.smithi122.stdout:3308 write 0x4088200 thru 0x409487b (0xc67c bytes) 2024-09-19T21:35:49.511 INFO:teuthology.orchestra.run.smithi122.stdout:3311 write 0xcc05c00 thru 0xcc075bb (0x19bc bytes) 2024-09-19T21:35:49.511 INFO:teuthology.orchestra.run.smithi122.stdout:3313 punch from 0x7ccaa00 to 0x7ccce00, (0x2400 bytes) 2024-09-19T21:35:49.520 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:49.526 INFO:teuthology.orchestra.run.smithi122.stdout:3315 trunc from 0xe52a000 to 0x866f800 2024-09-19T21:35:49.541 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:49.562 INFO:teuthology.orchestra.run.smithi122.stdout:3316 read 0x8089c00 thru 0x8095fd1 (0xc3d2 bytes) 2024-09-19T21:35:49.569 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:49.571 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:49.592 INFO:teuthology.orchestra.run.smithi122.stdout:3317 write 0xcaba800 thru 0xcabaff7 (0x7f8 bytes) 2024-09-19T21:35:49.594 INFO:teuthology.orchestra.run.smithi122.stdout:3318 write 0xa723200 thru 0xa732387 (0xf188 bytes) 2024-09-19T21:35:49.594 INFO:teuthology.orchestra.run.smithi122.stdout:3320 punch from 0x22b2800 to 0x22bb600, (0x8e00 bytes) 2024-09-19T21:35:49.595 INFO:teuthology.orchestra.run.smithi122.stdout:3321 read 0x3f87000 thru 0x3f8f9c5 (0x89c6 bytes) 2024-09-19T21:35:49.595 INFO:teuthology.orchestra.run.smithi122.stdout:3325 read 0x705a200 thru 0x705dc2f (0x3a30 bytes) 2024-09-19T21:35:49.595 INFO:teuthology.orchestra.run.smithi122.stdout:3328 write 0xdf0200 thru 0xdf8654 (0x8455 bytes) 2024-09-19T21:35:49.595 INFO:teuthology.orchestra.run.smithi122.stdout:3337 write 0xbb7400 thru 0xbb8ce2 (0x18e3 bytes) 2024-09-19T21:35:49.596 INFO:teuthology.orchestra.run.smithi122.stdout:3339 write 0xb984a00 thru 0xb989d30 (0x5331 bytes) 2024-09-19T21:35:49.596 INFO:teuthology.orchestra.run.smithi122.stdout:3340 read 0x98be800 thru 0x98c12d1 (0x2ad2 bytes) 2024-09-19T21:35:49.597 INFO:teuthology.orchestra.run.smithi122.stdout:3341 write 0x8c88000 thru 0x8c8d4ff (0x5500 bytes) 2024-09-19T21:35:49.597 INFO:teuthology.orchestra.run.smithi122.stdout:3342 write 0x8fa4c00 thru 0x8fb3ffe (0xf3ff bytes) 2024-09-19T21:35:49.598 INFO:teuthology.orchestra.run.smithi122.stdout:3343 read 0x400d800 thru 0x4018eff (0xb700 bytes) 2024-09-19T21:35:49.599 INFO:teuthology.orchestra.run.smithi122.stdout:3344 punch from 0x5d5a000 to 0x5d5ae00, (0xe00 bytes) 2024-09-19T21:35:49.601 INFO:teuthology.orchestra.run.smithi122.stdout:3345 punch from 0x1c17000 to 0x1c22200, (0xb200 bytes) 2024-09-19T21:35:49.601 INFO:teuthology.orchestra.run.smithi122.stdout:3347 write 0x9684e00 thru 0x968640f (0x1610 bytes) 2024-09-19T21:35:49.601 INFO:teuthology.orchestra.run.smithi122.stdout:3349 punch from 0xa9b5200 to 0xa9c3e00, (0xec00 bytes) 2024-09-19T21:35:49.601 INFO:teuthology.orchestra.run.smithi122.stdout:3350 read 0x4273600 thru 0x427816e (0x4b6f bytes) 2024-09-19T21:35:49.606 INFO:teuthology.orchestra.run.smithi122.stdout:3352 trunc from 0xcabb000 to 0xd733c00 2024-09-19T21:35:49.613 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:49.613 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:49.613 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:49.613 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:49.613 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:49.613 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:49.606+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:49.623 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:49.624 INFO:teuthology.orchestra.run.smithi122.stdout:3355 read 0x172a200 thru 0x1733b59 (0x995a bytes) 2024-09-19T21:35:49.628 INFO:teuthology.orchestra.run.smithi122.stdout:3356 punch from 0x521d200 to 0x5225400, (0x8200 bytes) 2024-09-19T21:35:49.628 INFO:teuthology.orchestra.run.smithi122.stdout:3358 read 0xa531200 thru 0xa536ff8 (0x5df9 bytes) 2024-09-19T21:35:49.657 INFO:teuthology.orchestra.run.smithi122.stdout:3360 trunc from 0xd733c00 to 0x635000 2024-09-19T21:35:49.698 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:49.690+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.047045945000001, 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:49.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:49.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4450907009999998, 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:35:49.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:35:49.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:35:49.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:35:49.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4446783540000001, 2024-09-19T21:35:49.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:35:49.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4446614520000001, 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:35:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:35:49.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4446447190000002, 2024-09-19T21:35:49.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:35:49.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:35:49.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4445636670000002, 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:35:49.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:35:49.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4445469370000001, 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.815 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4445305400000001, 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:49.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:49.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:35:49.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.822 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4445141069999998, 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4444985510000001, 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:35:49.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:35:49.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4443844820000002, 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:35:49.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.837 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.444367519, 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.840 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:35:49.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.444351535, 2024-09-19T21:35:49.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.44433436, 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.847 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:49.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:35:49.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4443171619999999, 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.852 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:49.856 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.444300331, 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:35:49.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.861 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4442838099999999, 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.864 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:35:49.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:49.866 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4442612060000002, 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:35:49.870 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.444244818, 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:35:49.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:35:49.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:35:49.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.444135937, 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:49.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:49.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:49.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:49.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:49.879 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3361 punch from 0x303e00 to 0x30f400, (0xb600 bytes) 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3362 read 0x481c00 thru 0x49012a (0xe52b bytes) 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3366 read 0x40ce00 thru 0x41ad0a (0xdf0b bytes) 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3367 write 0x71a400 thru 0x71b629 (0x122a bytes) 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3368 write 0x6df4e00 thru 0x6df7683 (0x2884 bytes) 2024-09-19T21:35:49.881 INFO:teuthology.orchestra.run.smithi122.stdout:3371 trunc from 0x6df7800 to 0x6554800 2024-09-19T21:35:49.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:49.882 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:49.876+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:49.922 INFO:teuthology.orchestra.run.smithi122.stdout:3374 write 0xb54d600 thru 0xb55a866 (0xd267 bytes) 2024-09-19T21:35:49.923 INFO:teuthology.orchestra.run.smithi122.stdout:3375 write 0x8ecf000 thru 0x8ed828c (0x928d bytes) 2024-09-19T21:35:49.923 INFO:teuthology.orchestra.run.smithi122.stdout:3380 write 0x1147400 thru 0x11497b2 (0x23b3 bytes) 2024-09-19T21:35:49.958 INFO:teuthology.orchestra.run.smithi122.stdout:3381 write 0xd839a00 thru 0xd83a1cd (0x7ce bytes) 2024-09-19T21:35:49.960 INFO:teuthology.orchestra.run.smithi122.stdout:3382 read 0x1e98200 thru 0x1ea3354 (0xb155 bytes) 2024-09-19T21:35:49.961 INFO:teuthology.orchestra.run.smithi122.stdout:3384 read 0x4043400 thru 0x404391e (0x51f bytes) 2024-09-19T21:35:49.962 INFO:teuthology.orchestra.run.smithi122.stdout:3389 write 0x8c6d200 thru 0x8c71a5d (0x485e bytes) 2024-09-19T21:35:49.962 INFO:teuthology.orchestra.run.smithi122.stdout:3390 write 0xd804200 thru 0xd80b374 (0x7175 bytes) 2024-09-19T21:35:49.962 INFO:teuthology.orchestra.run.smithi122.stdout:3391 write 0x402b000 thru 0x403494a (0x994b bytes) 2024-09-19T21:35:49.963 INFO:teuthology.orchestra.run.smithi122.stdout:3393 read 0xaffdc00 thru 0xb008074 (0xa475 bytes) 2024-09-19T21:35:49.964 INFO:teuthology.orchestra.run.smithi122.stdout:3394 write 0xb76b800 thru 0xb778fdc (0xd7dd bytes) 2024-09-19T21:35:49.965 INFO:teuthology.orchestra.run.smithi122.stdout:3396 read 0x53e7c00 thru 0x53f4805 (0xcc06 bytes) 2024-09-19T21:35:49.966 INFO:teuthology.orchestra.run.smithi122.stdout:3397 punch from 0xcafba00 to 0xcb01c00, (0x6200 bytes) 2024-09-19T21:35:49.967 INFO:teuthology.orchestra.run.smithi122.stdout:3398 punch from 0x95e3400 to 0x95ece00, (0x9a00 bytes) 2024-09-19T21:35:49.968 INFO:teuthology.orchestra.run.smithi122.stdout:3399 read 0xd572800 thru 0xd57b03e (0x883f bytes) 2024-09-19T21:35:49.976 INFO:teuthology.orchestra.run.smithi122.stdout:3401 trunc from 0xd83a200 to 0xa0ec600 2024-09-19T21:35:49.982 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:49.974+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:50.011 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.011 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:50.011 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:50.011 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.011 INFO:teuthology.orchestra.run.smithi122.stdout:3404 read 0x2cfb800 thru 0x2d01884 (0x6085 bytes) 2024-09-19T21:35:50.017 INFO:teuthology.orchestra.run.smithi122.stdout:3406 write 0x62b4a00 thru 0x62b4c6a (0x26b bytes) 2024-09-19T21:35:50.017 INFO:teuthology.orchestra.run.smithi122.stdout:3407 read 0x33fda00 thru 0x340698e (0x8f8f bytes) 2024-09-19T21:35:50.019 INFO:teuthology.orchestra.run.smithi122.stdout:3415 read 0x686d800 thru 0x6877bf7 (0xa3f8 bytes) 2024-09-19T21:35:50.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:50.052 INFO:teuthology.orchestra.run.smithi122.stdout:3416 write 0xedb1e00 thru 0xedbaea4 (0x90a5 bytes) 2024-09-19T21:35:50.053 INFO:teuthology.orchestra.run.smithi122.stdout:3417 write 0xe0be600 thru 0xe0c848a (0x9e8b bytes) 2024-09-19T21:35:50.053 INFO:teuthology.orchestra.run.smithi122.stdout:3418 read 0xd536a00 thru 0xd53b7ab (0x4dac bytes) 2024-09-19T21:35:50.067 INFO:teuthology.orchestra.run.smithi122.stdout:3421 trunc from 0xedbb000 to 0x80f9600 2024-09-19T21:35:50.082 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:50.074+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:50.106 INFO:teuthology.orchestra.run.smithi122.stdout:3422 read 0x2de0200 thru 0x2de5c2d (0x5a2e bytes) 2024-09-19T21:35:50.112 INFO:teuthology.orchestra.run.smithi122.stdout:3423 punch from 0x27e8e00 to 0x27f1000, (0x8200 bytes) 2024-09-19T21:35:50.142 INFO:teuthology.orchestra.run.smithi122.stdout:3426 write 0xca0ae00 thru 0xca145bc (0x97bd bytes) 2024-09-19T21:35:50.143 INFO:teuthology.orchestra.run.smithi122.stdout:3429 write 0x3a8a400 thru 0x3a954b9 (0xb0ba bytes) 2024-09-19T21:35:50.144 INFO:teuthology.orchestra.run.smithi122.stdout:3431 read 0x4a49600 thru 0x4a52743 (0x9144 bytes) 2024-09-19T21:35:50.144 INFO:teuthology.orchestra.run.smithi122.stdout:3434 write 0x213da00 thru 0x214c9aa (0xefab bytes) 2024-09-19T21:35:50.145 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.145 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:50.145 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:50.145 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:50.145 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:50.157 INFO:teuthology.orchestra.run.smithi122.stdout:3436 trunc from 0xca14600 to 0x741bc00 2024-09-19T21:35:50.183 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:50.176+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:50.201 INFO:teuthology.orchestra.run.smithi122.stdout:3440 read 0xbd2a00 thru 0xbde4d2 (0xbad3 bytes) 2024-09-19T21:35:50.208 INFO:teuthology.orchestra.run.smithi122.stdout:3441 trunc from 0x741bc00 to 0x9267e00 2024-09-19T21:35:50.230 INFO:teuthology.orchestra.run.smithi122.stdout:3443 write 0x71f8600 thru 0x71ff4cf (0x6ed0 bytes) 2024-09-19T21:35:50.235 INFO:teuthology.orchestra.run.smithi122.stdout:3444 read 0x88df800 thru 0x88e129e (0x1a9f bytes) 2024-09-19T21:35:50.235 INFO:teuthology.orchestra.run.smithi122.stdout:3446 punch from 0x5dda600 to 0x5de6800, (0xc200 bytes) 2024-09-19T21:35:50.236 INFO:teuthology.orchestra.run.smithi122.stdout:3447 read 0x8feee00 thru 0x8ff4881 (0x5a82 bytes) 2024-09-19T21:35:50.237 INFO:teuthology.orchestra.run.smithi122.stdout:3449 punch from 0x3f1bc00 to 0x3f20200, (0x4600 bytes) 2024-09-19T21:35:50.237 INFO:teuthology.orchestra.run.smithi122.stdout:3451 read 0x2f8ba00 thru 0x2f94a57 (0x9058 bytes) 2024-09-19T21:35:50.241 INFO:teuthology.orchestra.run.smithi122.stdout:3452 punch from 0x23c0a00 to 0x23c2a00, (0x2000 bytes) 2024-09-19T21:35:50.270 INFO:teuthology.orchestra.run.smithi122.stdout:3454 write 0xc7a4600 thru 0xc7b3565 (0xef66 bytes) 2024-09-19T21:35:50.272 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.272 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:50.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015325126999997, 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:50.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:50.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:50.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015311588000003, 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:50.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015285847999998, 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015271634000001, 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:50.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015250717000001, 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015235341, 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:50.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:50.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015220413000002, 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:50.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:50.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015167793000003, 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015118215000001, 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015101000000001, 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:50.320 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015081862000002, 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:50.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:50.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015067014000003, 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:50.329 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.015044021000001, 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:50.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.014311728999999, 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.334 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:50.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:50.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.014261419, 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.338 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.014199107000003, 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.012868474000001, 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:50.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.012732896000003, 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.356 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.012656137, 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:50.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.012597413000002, 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3456 read 0x6a24400 thru 0x6a28e85 (0x4a86 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3457 read 0x88bb000 thru 0x88c31cd (0x81ce bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3458 read 0x6e5de00 thru 0x6e622f1 (0x44f2 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3459 read 0x6253a00 thru 0x6258858 (0x4e59 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3462 read 0x2d01600 thru 0x2d0dc83 (0xc684 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3464 punch from 0xa2afc00 to 0xa2bdc00, (0xe000 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3465 punch from 0x8b8a00 to 0x8c3800, (0xae00 bytes) 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3466 trunc from 0xc7b3600 to 0x93c3a00 2024-09-19T21:35:50.368 INFO:teuthology.orchestra.run.smithi122.stdout:3467 read 0x8c4e400 thru 0x8c4f2ba (0xebb bytes) 2024-09-19T21:35:50.369 INFO:teuthology.orchestra.run.smithi122.stdout:3468 trunc from 0x93c3a00 to 0x6638e00 2024-09-19T21:35:50.369 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:50.358+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:50.387 INFO:teuthology.orchestra.run.smithi122.stdout:3472 write 0xd8e1000 thru 0xd8f081c (0xf81d bytes) 2024-09-19T21:35:50.388 INFO:teuthology.orchestra.run.smithi122.stdout:3474 punch from 0x3da3600 to 0x3da6800, (0x3200 bytes) 2024-09-19T21:35:50.389 INFO:teuthology.orchestra.run.smithi122.stdout:3476 punch from 0xc056200 to 0xc061000, (0xae00 bytes) 2024-09-19T21:35:50.389 INFO:teuthology.orchestra.run.smithi122.stdout:3477 write 0x39e6400 thru 0x39f5f81 (0xfb82 bytes) 2024-09-19T21:35:50.392 INFO:teuthology.orchestra.run.smithi122.stdout:3480 punch from 0x1d56c00 to 0x1d65e00, (0xf200 bytes) 2024-09-19T21:35:50.414 INFO:teuthology.orchestra.run.smithi122.stdout:3482 trunc from 0xd8f0a00 to 0x227b800 2024-09-19T21:35:50.467 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:50.460+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:50.493 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.494 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:50.494 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:50.494 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.500 INFO:teuthology.orchestra.run.smithi122.stdout:3483 write 0x9cb4000 thru 0x9cc2efc (0xeefd bytes) 2024-09-19T21:35:50.504 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:50.506 INFO:teuthology.orchestra.run.smithi122.stdout:3484 punch from 0x6007c00 to 0x600da00, (0x5e00 bytes) 2024-09-19T21:35:50.520 INFO:teuthology.orchestra.run.smithi122.stdout:3487 trunc from 0x9cc3000 to 0x4999a00 2024-09-19T21:35:50.551 INFO:teuthology.orchestra.run.smithi122.stdout:3488 read 0xc28800 thru 0xc35f07 (0xd708 bytes) 2024-09-19T21:35:50.569 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:50.564+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:50.585 INFO:teuthology.orchestra.run.smithi122.stdout:3489 write 0x7eb7800 thru 0x7eb8c12 (0x1413 bytes) 2024-09-19T21:35:50.588 INFO:teuthology.orchestra.run.smithi122.stdout:3490 punch from 0xb30000 to 0xb37c00, (0x7c00 bytes) 2024-09-19T21:35:50.589 INFO:teuthology.orchestra.run.smithi122.stdout:3492 read 0x3a6ae00 thru 0x3a77703 (0xc904 bytes) 2024-09-19T21:35:50.590 INFO:teuthology.orchestra.run.smithi122.stdout:3493 write 0x5983600 thru 0x598abf2 (0x75f3 bytes) 2024-09-19T21:35:50.593 INFO:teuthology.orchestra.run.smithi122.stdout:3495 punch from 0xd37600 to 0xd43600, (0xc000 bytes) 2024-09-19T21:35:50.594 INFO:teuthology.orchestra.run.smithi122.stdout:3496 write 0x12ad800 thru 0x12b367c (0x5e7d bytes) 2024-09-19T21:35:50.613 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:50.614 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:50.629 INFO:teuthology.orchestra.run.smithi122.stdout:3497 write 0xc48fc00 thru 0xc499d45 (0xa146 bytes) 2024-09-19T21:35:50.630 INFO:teuthology.orchestra.run.smithi122.stdout:3498 read 0x3c15800 thru 0x3c196a7 (0x3ea8 bytes) 2024-09-19T21:35:50.631 INFO:teuthology.orchestra.run.smithi122.stdout:3499 read 0x381da00 thru 0x3828295 (0xa896 bytes) 2024-09-19T21:35:50.632 INFO:teuthology.orchestra.run.smithi122.stdout:3500 read 0xbd00c00 thru 0xbd0b414 (0xa815 bytes) 2024-09-19T21:35:50.633 INFO:teuthology.orchestra.run.smithi122.stdout:3506 write 0x1048000 thru 0x104a9ea (0x29eb bytes) 2024-09-19T21:35:50.634 INFO:teuthology.orchestra.run.smithi122.stdout:3507 write 0x1e23a00 thru 0x1e24ed2 (0x14d3 bytes) 2024-09-19T21:35:50.635 INFO:teuthology.orchestra.run.smithi122.stdout:3509 write 0x2152a00 thru 0x21619f5 (0xeff6 bytes) 2024-09-19T21:35:50.661 INFO:teuthology.orchestra.run.smithi122.stdout:3510 trunc from 0xc499e00 to 0x8f3a00 2024-09-19T21:35:50.669 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:50.662+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:50.770 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:50.764+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:50.777 INFO:teuthology.orchestra.run.smithi122.stdout:3512 read 0x260c00 thru 0x2652ec (0x46ed bytes) 2024-09-19T21:35:50.780 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.579208465000001, 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.785 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.578628554000002, 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:35:50.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:35:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:50.790 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.578565907000002, 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:50.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:35:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:35:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.795 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.578499587, 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:35:50.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577795908999999, 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:50.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:35:50.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577723086999999, 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:35:50.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577709348999999, 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:35:50.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.813 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577691659999999, 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:35:50.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577641274000001, 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:35:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577614385, 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:35:50.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577573809, 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.828 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.831 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577552194999999, 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:35:50.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:50.836 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.577532995999999, 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:35:50.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0888155019999992, 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.841 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:35:50.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0885482250000003, 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:35:50.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0882153940000006, 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0876682500000001, 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:35:50.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0875919120000006, 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:35:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:35:50.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0872596360000006, 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:50.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.868 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0869567100000008, 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:35:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:50.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:50.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:50.873 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:50.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:50.876 INFO:teuthology.orchestra.run.smithi122.stdout:3513 write 0x2e29600 thru 0x2e32609 (0x900a bytes) 2024-09-19T21:35:50.876 INFO:teuthology.orchestra.run.smithi122.stdout:3514 read 0x1318800 thru 0x1320627 (0x7e28 bytes) 2024-09-19T21:35:50.876 INFO:teuthology.orchestra.run.smithi122.stdout:3515 read 0x179800 thru 0x18542b (0xbc2c bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3516 read 0x2d5e00 thru 0x2df554 (0x9755 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3519 trunc from 0x2e32800 to 0x2ffb800 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3521 read 0x1173800 thru 0x117cd03 (0x9504 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3523 write 0x2d69200 thru 0x2d69408 (0x209 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3524 write 0x2a54c00 thru 0x2a614f3 (0xc8f4 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3526 write 0x2294c00 thru 0x22989f8 (0x3df9 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3527 punch from 0x144a000 to 0x1454600, (0xa600 bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3528 write 0x29fb400 thru 0x2a046bc (0x92bd bytes) 2024-09-19T21:35:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:3530 trunc from 0x2ffb800 to 0xeb2400 2024-09-19T21:35:50.878 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:50.866+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:50.958 INFO:teuthology.orchestra.run.smithi122.stdout:3531 write 0xd4a9800 thru 0xd4aca55 (0x3256 bytes) 2024-09-19T21:35:50.975 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:50.968+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:50.980 INFO:teuthology.orchestra.run.smithi122.stdout:3532 trunc from 0xd4acc00 to 0x2da3a00 2024-09-19T21:35:50.995 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:50.995 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:51.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:51.035 INFO:teuthology.orchestra.run.smithi122.stdout:3533 read 0x19ffe00 thru 0x1a02bd3 (0x2dd4 bytes) 2024-09-19T21:35:51.065 INFO:teuthology.orchestra.run.smithi122.stdout:3535 write 0x3f7da00 thru 0x3f86031 (0x8632 bytes) 2024-09-19T21:35:51.076 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:51.068+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:51.100 INFO:teuthology.orchestra.run.smithi122.stdout:3538 write 0x551b000 thru 0x5522d4f (0x7d50 bytes) 2024-09-19T21:35:51.103 INFO:teuthology.orchestra.run.smithi122.stdout:3539 punch from 0x2edb400 to 0x2ee6200, (0xae00 bytes) 2024-09-19T21:35:51.111 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:51.111 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:51.111 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:51.111 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:51.111 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:51.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:51.129 INFO:teuthology.orchestra.run.smithi122.stdout:3540 write 0x6f91a00 thru 0x6f937f4 (0x1df5 bytes) 2024-09-19T21:35:51.134 INFO:teuthology.orchestra.run.smithi122.stdout:3541 read 0x2289200 thru 0x229397a (0xa77b bytes) 2024-09-19T21:35:51.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:35:51.163 INFO:teuthology.orchestra.run.smithi122.stdout:3542 write 0x7b03400 thru 0x7b0a461 (0x7062 bytes) 2024-09-19T21:35:51.165 INFO:teuthology.orchestra.run.smithi122.stdout:3543 read 0x195a00 thru 0x19ff49 (0xa54a bytes) 2024-09-19T21:35:51.168 INFO:teuthology.orchestra.run.smithi122.stdout:3544 read 0x1a4d200 thru 0x1a567bc (0x95bd bytes) 2024-09-19T21:35:51.179 INFO:teuthology.orchestra.run.smithi122.stdout:3546 trunc from 0x7b0a600 to 0x4071800 2024-09-19T21:35:51.203 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:51.198+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:51.214 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:51.214 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:51.215 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-19T21:35:51.215 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:35:51.215 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:35:51.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-19T21:35:51.243 INFO:teuthology.orchestra.run.smithi122.stdout:3547 write 0x8b2a400 thru 0x8b2dcf9 (0x38fa bytes) 2024-09-19T21:35:51.244 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.491911696999999, 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:51.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:51.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864572301000001, 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:35:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864519834999999, 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:35:51.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864489288, 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:35:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864466691000001, 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.265 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:35:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:35:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864444239000001, 2024-09-19T21:35:51.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:35:51.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864429942999999, 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:35:51.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864413316, 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.86439468, 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.283 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:35:51.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864368015, 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:35:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:35:51.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864357804000001, 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.293 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:35:51.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.864345318, 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.298 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:35:51.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860470727999999, 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.303 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:35:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860453208999999, 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.308 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:35:51.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.312 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860415655000001, 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:35:51.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860397812, 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860385363000001, 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:35:51.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860374671000001, 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.327 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:35:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.860363094, 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 10.444391138, 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:51.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:35:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:51.341 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:51.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3550 read 0x7567800 thru 0x756c79c (0x4f9d bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3551 punch from 0x1fce00 to 0x204400, (0x7600 bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3552 write 0xe6200 thru 0xec884 (0x6685 bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3553 read 0x456ec00 thru 0x4570c3b (0x203c bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3554 write 0xc102200 thru 0xc107756 (0x5557 bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3555 write 0x9563400 thru 0x956498b (0x158c bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3556 punch from 0x1b61a00 to 0x1b6d800, (0xbe00 bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3557 read 0x51e5e00 thru 0x51f3e4c (0xe04d bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3558 write 0xc579600 thru 0xc57d8bd (0x42be bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3560 write 0xbf64200 thru 0xbf6f60a (0xb40b bytes) 2024-09-19T21:35:51.345 INFO:teuthology.orchestra.run.smithi122.stdout:3561 trunc from 0xc57da00 to 0x7004800 2024-09-19T21:35:51.346 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T21:35:51.346 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:51.346 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:51.336+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:51.391 INFO:teuthology.orchestra.run.smithi122.stdout:3563 trunc from 0x7004800 to 0xe95b600 2024-09-19T21:35:51.409 INFO:teuthology.orchestra.run.smithi122.stdout:3564 read 0x923ae00 thru 0x92406c3 (0x58c4 bytes) 2024-09-19T21:35:51.411 INFO:teuthology.orchestra.run.smithi122.stdout:3566 read 0x8ad0200 thru 0x8ade6d3 (0xe4d4 bytes) 2024-09-19T21:35:51.418 INFO:teuthology.orchestra.run.smithi122.stdout:3568 trunc from 0xe95b600 to 0xc356000 2024-09-19T21:35:51.437 INFO:teuthology.orchestra.run.smithi122.stdout:3569 write 0x1508c00 thru 0x15157b6 (0xcbb7 bytes) 2024-09-19T21:35:51.443 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:51.438+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:51.446 INFO:teuthology.orchestra.run.smithi122.stdout:3571 trunc from 0xc356000 to 0x9839a00 2024-09-19T21:35:51.454 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:51.454 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:51.454 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:51.454 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:51.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:51.474 INFO:teuthology.orchestra.run.smithi122.stdout:3573 write 0x3080a00 thru 0x308ca5f (0xc060 bytes) 2024-09-19T21:35:51.504 INFO:teuthology.orchestra.run.smithi122.stdout:3574 write 0xbe5e000 thru 0xbe64ccb (0x6ccc bytes) 2024-09-19T21:35:51.532 INFO:teuthology.orchestra.run.smithi122.stdout:3575 write 0xca54800 thru 0xca58ce2 (0x44e3 bytes) 2024-09-19T21:35:51.536 INFO:teuthology.orchestra.run.smithi122.stdout:3576 trunc from 0xca58e00 to 0xd7db800 2024-09-19T21:35:51.543 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:51.538+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:51.570 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:51.570 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:51.570 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:51.570 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:51.571 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:51.571 INFO:teuthology.orchestra.run.smithi122.stdout:3578 trunc from 0xd7db800 to 0xaa96800 2024-09-19T21:35:51.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:51.583 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:51.609 INFO:teuthology.orchestra.run.smithi122.stdout:3579 write 0x4323c00 thru 0x4331a8a (0xde8b bytes) 2024-09-19T21:35:51.614 INFO:teuthology.orchestra.run.smithi122.stdout:3580 read 0x8407e00 thru 0x84173e8 (0xf5e9 bytes) 2024-09-19T21:35:51.618 INFO:teuthology.orchestra.run.smithi122.stdout:3583 read 0x85dd800 thru 0x85ec4df (0xece0 bytes) 2024-09-19T21:35:51.621 INFO:teuthology.orchestra.run.smithi122.stdout:3586 read 0x240c400 thru 0x241bd2b (0xf92c bytes) 2024-09-19T21:35:51.621 INFO:teuthology.orchestra.run.smithi122.stdout:3587 punch from 0x37eca00 to 0x37f3c00, (0x7200 bytes) 2024-09-19T21:35:51.622 INFO:teuthology.orchestra.run.smithi122.stdout:3588 punch from 0x43b800 to 0x441a00, (0x6200 bytes) 2024-09-19T21:35:51.644 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:51.640+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:51.646 INFO:teuthology.orchestra.run.smithi122.stdout:3591 write 0xcddbe00 thru 0xcddd84f (0x1a50 bytes) 2024-09-19T21:35:51.661 INFO:teuthology.orchestra.run.smithi122.stdout:3592 trunc from 0xcddda00 to 0x6b63800 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.691 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.331098778000001, 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:51.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.331086469999999, 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:51.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:51.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:51.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.298355230999999, 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:51.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:51.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:51.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068597537999999, 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:51.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:51.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068574103, 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:51.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068552374999999, 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068531643, 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:51.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068497207, 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:51.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:51.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.068454410000001, 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:51.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063228899999999, 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:51.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.739 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063206602000001, 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:51.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:51.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063175746999999, 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:51.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:51.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063133038, 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:51.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:51.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063109051000001, 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.757 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063077647, 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:51.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.761 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063052481, 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:51.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.063007403, 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:51.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.062980659000001, 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:51.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.774 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.062967366999999, 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:51.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:51.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.062950447999999, 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:51.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:51.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:51.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:51.782 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:51.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:51.783 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:51.785 INFO:teuthology.orchestra.run.smithi122.stdout:3593 trunc from 0x6b63800 to 0x6dfd000 2024-09-19T21:35:51.785 INFO:teuthology.orchestra.run.smithi122.stdout:3594 read 0xf0bc00 thru 0xf0d24f (0x1650 bytes) 2024-09-19T21:35:51.785 INFO:teuthology.orchestra.run.smithi122.stdout:3596 trunc from 0x6dfd000 to 0xd29a400 2024-09-19T21:35:51.785 INFO:teuthology.orchestra.run.smithi122.stdout:3599 read 0xb3c5a00 thru 0xb3c66ea (0xceb bytes) 2024-09-19T21:35:51.785 INFO:teuthology.orchestra.run.smithi122.stdout:3600 trunc from 0xd29a400 to 0xbfdc800 2024-09-19T21:35:51.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:51.786 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:51.787 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:51.787 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:51.778+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:51.796 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:51.797 INFO:teuthology.orchestra.run.smithi122.stdout:3602 punch from 0x4453000 to 0x445d400, (0xa400 bytes) 2024-09-19T21:35:51.797 INFO:teuthology.orchestra.run.smithi122.stdout:3603 write 0x5750200 thru 0x575fb42 (0xf943 bytes) 2024-09-19T21:35:51.804 INFO:teuthology.orchestra.run.smithi122.stdout:3604 trunc from 0xbfdc800 to 0x9c88a00 2024-09-19T21:35:51.819 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:51.833 INFO:teuthology.orchestra.run.smithi122.stdout:3605 write 0x5730000 thru 0x5736441 (0x6442 bytes) 2024-09-19T21:35:51.844 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:51.846 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:51.885 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:51.880+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:51.896 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:51.896 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:51.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:51.912 INFO:teuthology.orchestra.run.smithi122.stdout:3607 write 0xd9c2800 thru 0xd9c3b86 (0x1387 bytes) 2024-09-19T21:35:51.914 INFO:teuthology.orchestra.run.smithi122.stdout:3609 read 0xa1bec00 thru 0xa1c8e56 (0xa257 bytes) 2024-09-19T21:35:51.915 INFO:teuthology.orchestra.run.smithi122.stdout:3612 trunc from 0xd9c3c00 to 0xd936000 2024-09-19T21:35:51.947 INFO:teuthology.orchestra.run.smithi122.stdout:3613 punch from 0x9999800 to 0x999a600, (0xe00 bytes) 2024-09-19T21:35:51.948 INFO:teuthology.orchestra.run.smithi122.stdout:3616 punch from 0x4dccc00 to 0x4dd1c00, (0x5000 bytes) 2024-09-19T21:35:51.972 INFO:teuthology.orchestra.run.smithi122.stdout:3619 write 0xe9e5800 thru 0xe9ec399 (0x6b9a bytes) 2024-09-19T21:35:51.973 INFO:teuthology.orchestra.run.smithi122.stdout:3620 write 0xd57200 thru 0xd5aa55 (0x3856 bytes) 2024-09-19T21:35:51.974 INFO:teuthology.orchestra.run.smithi122.stdout:3625 write 0x862b200 thru 0x86306bf (0x54c0 bytes) 2024-09-19T21:35:51.985 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:51.980+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:52.006 INFO:teuthology.orchestra.run.smithi122.stdout:3626 trunc from 0xe9ec400 to 0xf1ea00 2024-09-19T21:35:52.012 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.013 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:52.086 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:52.078+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:52.131 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.131 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.145 INFO:teuthology.orchestra.run.smithi122.stdout:3628 write 0xd7cb000 thru 0xd7da3d8 (0xf3d9 bytes) 2024-09-19T21:35:52.147 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:52.171 INFO:teuthology.orchestra.run.smithi122.stdout:3629 trunc from 0xd7da400 to 0x16b4c00 2024-09-19T21:35:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:35:52.187 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:52.182+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:52.232 INFO:teuthology.orchestra.run.smithi122.stdout:3636 punch from 0x14ec00 to 0x154400, (0x5800 bytes) 2024-09-19T21:35:52.235 INFO:teuthology.orchestra.run.smithi122.stdout:3637 read 0x1639400 thru 0x1648186 (0xed87 bytes) 2024-09-19T21:35:52.242 INFO:teuthology.orchestra.run.smithi122.stdout:3639 read 0x10a1000 thru 0x10af31a (0xe31b bytes) 2024-09-19T21:35:52.242 INFO:teuthology.orchestra.run.smithi122.stdout:3641 read 0x13f8e00 thru 0x1406647 (0xd848 bytes) 2024-09-19T21:35:52.243 INFO:teuthology.orchestra.run.smithi122.stdout:3646 read 0xec6800 thru 0xed43a6 (0xdba7 bytes) 2024-09-19T21:35:52.243 INFO:teuthology.orchestra.run.smithi122.stdout:3647 read 0x112c600 thru 0x1139e8e (0xd88f bytes) 2024-09-19T21:35:52.243 INFO:teuthology.orchestra.run.smithi122.stdout:3648 read 0x36e800 thru 0x37c53a (0xdd3b bytes) 2024-09-19T21:35:52.259 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.259 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.268 INFO:teuthology.orchestra.run.smithi122.stdout:3650 trunc from 0x16b4c00 to 0xc19b800 2024-09-19T21:35:52.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:52.285 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xecde600 2024-09-19T21:35:52.289 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:52.282+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:52.290 INFO:teuthology.orchestra.run.smithi122.stdout:3651 trunc from 0xc19b800 to 0xecde600 2024-09-19T21:35:52.315 INFO:teuthology.orchestra.run.smithi122.stdout:3654 read 0x160ba00 thru 0x1615aad (0xa0ae bytes) 2024-09-19T21:35:52.319 INFO:teuthology.orchestra.run.smithi122.stdout:3655 write 0x7d20a00 thru 0x7d30984 (0xff85 bytes) 2024-09-19T21:35:52.320 INFO:teuthology.orchestra.run.smithi122.stdout:3656 punch from 0x932aa00 to 0x9330200, (0x5800 bytes) 2024-09-19T21:35:52.332 INFO:teuthology.orchestra.run.smithi122.stdout:3660 trunc from 0xecde600 to 0xa3f6a00 2024-09-19T21:35:52.381 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.381 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.385 INFO:teuthology.orchestra.run.smithi122.stdout:3662 write 0xd22a600 thru 0xd231a8a (0x748b bytes) 2024-09-19T21:35:52.386 INFO:teuthology.orchestra.run.smithi122.stdout:3663 read 0x8567400 thru 0x857350d (0xc10e bytes) 2024-09-19T21:35:52.389 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:52.384+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:52.389 INFO:teuthology.orchestra.run.smithi122.stdout:3666 read 0xae8dc00 thru 0xae97822 (0x9c23 bytes) 2024-09-19T21:35:52.390 INFO:teuthology.orchestra.run.smithi122.stdout:3667 read 0x2c89c00 thru 0x2c95e8d (0xc28e bytes) 2024-09-19T21:35:52.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:52.407 INFO:teuthology.orchestra.run.smithi122.stdout:3670 trunc from 0xd231c00 to 0x754b000 2024-09-19T21:35:52.481 INFO:teuthology.orchestra.run.smithi122.stdout:3673 write 0x5cea00 thru 0x5de530 (0xfb31 bytes) 2024-09-19T21:35:52.490 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:52.482+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:52.490 INFO:teuthology.orchestra.run.smithi122.stdout:3674 trunc from 0x754b000 to 0x997b200 2024-09-19T21:35:52.513 INFO:teuthology.orchestra.run.smithi122.stdout:3675 write 0x8e00800 thru 0x8e0ba08 (0xb209 bytes) 2024-09-19T21:35:52.516 INFO:teuthology.orchestra.run.smithi122.stdout:3676 write 0x793f400 thru 0x794bc46 (0xc847 bytes) 2024-09-19T21:35:52.543 INFO:teuthology.orchestra.run.smithi122.stdout:3677 write 0xabf3800 thru 0xac01005 (0xd806 bytes) 2024-09-19T21:35:52.550 INFO:teuthology.orchestra.run.smithi122.stdout:3678 trunc from 0xac01200 to 0x8a68e00 2024-09-19T21:35:52.560 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.561 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:52.586 INFO:teuthology.orchestra.run.smithi122.stdout:3680 trunc from 0x8a68e00 to 0x7b68c00 2024-09-19T21:35:52.591 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:52.586+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:52.611 INFO:teuthology.orchestra.run.smithi122.stdout:3683 write 0x572e000 thru 0x573313b (0x513c bytes) 2024-09-19T21:35:52.613 INFO:teuthology.orchestra.run.smithi122.stdout:3684 punch from 0x48dc400 to 0x48e8a00, (0xc600 bytes) 2024-09-19T21:35:52.613 INFO:teuthology.orchestra.run.smithi122.stdout:3685 read 0xf15000 thru 0xf22fae (0xdfaf bytes) 2024-09-19T21:35:52.615 INFO:teuthology.orchestra.run.smithi122.stdout:3686 read 0x5323200 thru 0x5325c52 (0x2a53 bytes) 2024-09-19T21:35:52.616 INFO:teuthology.orchestra.run.smithi122.stdout:3690 read 0x6d19400 thru 0x6d27860 (0xe461 bytes) 2024-09-19T21:35:52.616 INFO:teuthology.orchestra.run.smithi122.stdout:3693 read 0x5533c00 thru 0x55410fb (0xd4fc bytes) 2024-09-19T21:35:52.617 INFO:teuthology.orchestra.run.smithi122.stdout:3695 read 0x4560c00 thru 0x4567dbc (0x71bd bytes) 2024-09-19T21:35:52.619 INFO:teuthology.orchestra.run.smithi122.stdout:3700 write 0x13c3a00 thru 0x13cf08f (0xb690 bytes) 2024-09-19T21:35:52.654 INFO:teuthology.orchestra.run.smithi122.stdout:3701 write 0xc566800 thru 0xc572278 (0xba79 bytes) 2024-09-19T21:35:52.655 INFO:teuthology.orchestra.run.smithi122.stdout:3702 read 0x4c9e000 thru 0x4ca8b36 (0xab37 bytes) 2024-09-19T21:35:52.657 INFO:teuthology.orchestra.run.smithi122.stdout:3705 write 0x25f2600 thru 0x25f3c1b (0x161c bytes) 2024-09-19T21:35:52.658 INFO:teuthology.orchestra.run.smithi122.stdout:3708 punch from 0x7710a00 to 0x7719e00, (0x9400 bytes) 2024-09-19T21:35:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:3712 read 0x7f21400 thru 0x7f30da8 (0xf9a9 bytes) 2024-09-19T21:35:52.660 INFO:teuthology.orchestra.run.smithi122.stdout:3713 read 0x96fa000 thru 0x9704a9f (0xaaa0 bytes) 2024-09-19T21:35:52.661 INFO:teuthology.orchestra.run.smithi122.stdout:3714 write 0x5e91800 thru 0x5ea01b1 (0xe9b2 bytes) 2024-09-19T21:35:52.662 INFO:teuthology.orchestra.run.smithi122.stdout:3715 punch from 0xded400 to 0xdf8200, (0xae00 bytes) 2024-09-19T21:35:52.663 INFO:teuthology.orchestra.run.smithi122.stdout:3716 write 0x637ca00 thru 0x6389b32 (0xd133 bytes) 2024-09-19T21:35:52.665 INFO:teuthology.orchestra.run.smithi122.stdout:3718 read 0xa23dc00 thru 0xa248b62 (0xaf63 bytes) 2024-09-19T21:35:52.686 INFO:teuthology.orchestra.run.smithi122.stdout:3719 trunc from 0xc572400 to 0x1e68000 2024-09-19T21:35:52.691 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:52.684+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:52.694 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.694 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:52.772 INFO:teuthology.orchestra.run.smithi122.stdout:3720 trunc from 0x1e68000 to 0xc6ef400 2024-09-19T21:35:52.793 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:52.788+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:52.797 INFO:teuthology.orchestra.run.smithi122.stdout:3721 punch from 0xc697800 to 0xc69a600, (0x2e00 bytes) 2024-09-19T21:35:52.816 INFO:teuthology.orchestra.run.smithi122.stdout:3723 trunc from 0xc6ef400 to 0x2c44400 2024-09-19T21:35:52.819 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.819 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:52.863 INFO:teuthology.orchestra.run.smithi122.stdout:3728 write 0x41f000 thru 0x421ba4 (0x2ba5 bytes) 2024-09-19T21:35:52.893 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:52.888+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:52.905 INFO:teuthology.orchestra.run.smithi122.stdout:3733 write 0x95c2000 thru 0x95c4457 (0x2458 bytes) 2024-09-19T21:35:52.913 INFO:teuthology.orchestra.run.smithi122.stdout:3736 trunc from 0x95c4600 to 0x72f8200 2024-09-19T21:35:52.983 INFO:teuthology.orchestra.run.smithi122.stdout:3737 punch from 0x19b7a00 to 0x19c0000, (0x8600 bytes) 2024-09-19T21:35:52.984 INFO:teuthology.orchestra.run.smithi122.stdout:3739 write 0x524e000 thru 0x5255bce (0x7bcf bytes) 2024-09-19T21:35:52.986 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:52.986 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:52.994 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:52.988+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:53.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:53.021 INFO:teuthology.orchestra.run.smithi122.stdout:3740 write 0xa8a1a00 thru 0xa8adaf2 (0xc0f3 bytes) 2024-09-19T21:35:53.023 INFO:teuthology.orchestra.run.smithi122.stdout:3743 write 0x5602e00 thru 0x5606742 (0x3943 bytes) 2024-09-19T21:35:53.024 INFO:teuthology.orchestra.run.smithi122.stdout:3744 read 0x712f800 thru 0x7137b87 (0x8388 bytes) 2024-09-19T21:35:53.027 INFO:teuthology.orchestra.run.smithi122.stdout:3746 read 0x1232600 thru 0x1234d7b (0x277c bytes) 2024-09-19T21:35:53.034 INFO:teuthology.orchestra.run.smithi122.stdout:3748 trunc from 0xa8adc00 to 0x866c200 2024-09-19T21:35:53.065 INFO:teuthology.orchestra.run.smithi122.stdout:3750 punch from 0x1c82200 to 0x1c84a00, (0x2800 bytes) 2024-09-19T21:35:53.095 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:53.090+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:53.097 INFO:teuthology.orchestra.run.smithi122.stdout:3751 write 0xad13e00 thru 0xad1f0b0 (0xb2b1 bytes) 2024-09-19T21:35:53.099 INFO:teuthology.orchestra.run.smithi122.stdout:3754 write 0x29b5600 thru 0x29bb758 (0x6159 bytes) 2024-09-19T21:35:53.099 INFO:teuthology.orchestra.run.smithi122.stdout:3755 write 0x18f2400 thru 0x18f850b (0x610c bytes) 2024-09-19T21:35:53.103 INFO:teuthology.orchestra.run.smithi122.stdout:3756 read 0x5912a00 thru 0x5916648 (0x3c49 bytes) 2024-09-19T21:35:53.111 INFO:teuthology.orchestra.run.smithi122.stdout:3757 trunc from 0xad1f200 to 0x8dc4600 2024-09-19T21:35:53.141 INFO:teuthology.orchestra.run.smithi122.stdout:3758 read 0x83d1600 thru 0x83d211a (0xb1b bytes) 2024-09-19T21:35:53.144 INFO:teuthology.orchestra.run.smithi122.stdout:3761 write 0x3fb4600 thru 0x3fbcb77 (0x8578 bytes) 2024-09-19T21:35:53.155 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.155 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:53.180 INFO:teuthology.orchestra.run.smithi122.stdout:3763 write 0xcbcb600 thru 0xcbd3527 (0x7f28 bytes) 2024-09-19T21:35:53.183 INFO:teuthology.orchestra.run.smithi122.stdout:3765 read 0xcda800 thru 0xcdf5c8 (0x4dc9 bytes) 2024-09-19T21:35:53.196 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:53.188+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:3766 trunc from 0xcbd3600 to 0x5868c00 2024-09-19T21:35:53.253 INFO:teuthology.orchestra.run.smithi122.stdout:3768 punch from 0x4fec200 to 0x4ff4800, (0x8600 bytes) 2024-09-19T21:35:53.255 INFO:teuthology.orchestra.run.smithi122.stdout:3771 punch from 0x470b200 to 0x4716400, (0xb200 bytes) 2024-09-19T21:35:53.258 INFO:teuthology.orchestra.run.smithi122.stdout:3772 trunc from 0x5868c00 to 0x6873400 2024-09-19T21:35:53.278 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.278 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.279 INFO:teuthology.orchestra.run.smithi122.stdout:3775 punch from 0x974200 to 0x97f000, (0xae00 bytes) 2024-09-19T21:35:53.280 INFO:teuthology.orchestra.run.smithi122.stdout:3779 read 0x2499a00 thru 0x24a4693 (0xac94 bytes) 2024-09-19T21:35:53.282 INFO:teuthology.orchestra.run.smithi122.stdout:3780 write 0x418f000 thru 0x4199b7a (0xab7b bytes) 2024-09-19T21:35:53.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:53.297 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:53.290+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:53.318 INFO:teuthology.orchestra.run.smithi122.stdout:3786 write 0xbcbda00 thru 0xbcc40f0 (0x66f1 bytes) 2024-09-19T21:35:53.324 INFO:teuthology.orchestra.run.smithi122.stdout:3787 trunc from 0xbcc4200 to 0x9904800 2024-09-19T21:35:53.370 INFO:teuthology.orchestra.run.smithi122.stdout:3788 trunc from 0x9904800 to 0x7513600 2024-09-19T21:35:53.396 INFO:teuthology.orchestra.run.smithi122.stdout:3789 punch from 0xd91c00 to 0xd9f400, (0xd800 bytes) 2024-09-19T21:35:53.398 INFO:teuthology.orchestra.run.smithi122.stdout:3790 punch from 0x461ce00 to 0x4622800, (0x5a00 bytes) 2024-09-19T21:35:53.398 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:53.390+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:53.398 INFO:teuthology.orchestra.run.smithi122.stdout:3793 trunc from 0x7513600 to 0x75b7800 2024-09-19T21:35:53.421 INFO:teuthology.orchestra.run.smithi122.stdout:3795 punch from 0x4ad3600 to 0x4ada000, (0x6a00 bytes) 2024-09-19T21:35:53.448 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.448 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:53.459 INFO:teuthology.orchestra.run.smithi122.stdout:3796 write 0xb970400 thru 0xb971c21 (0x1822 bytes) 2024-09-19T21:35:53.496 INFO:teuthology.orchestra.run.smithi122.stdout:3797 write 0xe5f0000 thru 0xe5f9a0c (0x9a0d bytes) 2024-09-19T21:35:53.499 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:53.492+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:53.510 INFO:teuthology.orchestra.run.smithi122.stdout:3799 trunc from 0xe5f9c00 to 0xa1c4400 2024-09-19T21:35:53.551 INFO:teuthology.orchestra.run.smithi122.stdout:3803 punch from 0x5c65200 to 0x5c6c400, (0x7200 bytes) 2024-09-19T21:35:53.554 INFO:teuthology.orchestra.run.smithi122.stdout:3807 read 0x1512600 thru 0x1518103 (0x5b04 bytes) 2024-09-19T21:35:53.555 INFO:teuthology.orchestra.run.smithi122.stdout:3808 write 0x735f400 thru 0x73650c1 (0x5cc2 bytes) 2024-09-19T21:35:53.565 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.565 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:53.588 INFO:teuthology.orchestra.run.smithi122.stdout:3810 write 0xdc0c000 thru 0xdc13ea0 (0x7ea1 bytes) 2024-09-19T21:35:53.590 INFO:teuthology.orchestra.run.smithi122.stdout:3812 write 0x3d72600 thru 0x3d7a042 (0x7a43 bytes) 2024-09-19T21:35:53.591 INFO:teuthology.orchestra.run.smithi122.stdout:3813 read 0xb3d5e00 thru 0xb3e0204 (0xa405 bytes) 2024-09-19T21:35:53.592 INFO:teuthology.orchestra.run.smithi122.stdout:3814 write 0x3b65800 thru 0x3b75616 (0xfe17 bytes) 2024-09-19T21:35:53.593 INFO:teuthology.orchestra.run.smithi122.stdout:3815 write 0x1970a00 thru 0x1970b6b (0x16c bytes) 2024-09-19T21:35:53.593 INFO:teuthology.orchestra.run.smithi122.stdout:3816 read 0x1fffc00 thru 0x1fffcd1 (0xd2 bytes) 2024-09-19T21:35:53.595 INFO:teuthology.orchestra.run.smithi122.stdout:3818 read 0x6d2b400 thru 0x6d309ba (0x55bb bytes) 2024-09-19T21:35:53.596 INFO:teuthology.orchestra.run.smithi122.stdout:3819 read 0x4098400 thru 0x40a4fd7 (0xcbd8 bytes) 2024-09-19T21:35:53.598 INFO:teuthology.orchestra.run.smithi122.stdout:3822 read 0xdb3f800 thru 0xdb46f44 (0x7745 bytes) 2024-09-19T21:35:53.598 INFO:teuthology.orchestra.run.smithi122.stdout:3824 read 0xcc30600 thru 0xcc3c7e7 (0xc1e8 bytes) 2024-09-19T21:35:53.598 INFO:teuthology.orchestra.run.smithi122.stdout:3825 write 0x45d3a00 thru 0x45e1f8e (0xe58f bytes) 2024-09-19T21:35:53.599 INFO:teuthology.orchestra.run.smithi122.stdout:3826 read 0x75b2000 thru 0x75b72a0 (0x52a1 bytes) 2024-09-19T21:35:53.599 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:53.594+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:53.601 INFO:teuthology.orchestra.run.smithi122.stdout:3828 trunc from 0xdc14000 to 0xdfc3e00 2024-09-19T21:35:53.631 INFO:teuthology.orchestra.run.smithi122.stdout:3829 write 0x8b75a00 thru 0x8b779c3 (0x1fc4 bytes) 2024-09-19T21:35:53.633 INFO:teuthology.orchestra.run.smithi122.stdout:3832 punch from 0xb0fc600 to 0xb0ffc00, (0x3600 bytes) 2024-09-19T21:35:53.634 INFO:teuthology.orchestra.run.smithi122.stdout:3837 punch from 0x2de7600 to 0x2de9c00, (0x2600 bytes) 2024-09-19T21:35:53.634 INFO:teuthology.orchestra.run.smithi122.stdout:3838 punch from 0x33f6200 to 0x3401a00, (0xb800 bytes) 2024-09-19T21:35:53.634 INFO:teuthology.orchestra.run.smithi122.stdout:3839 punch from 0x1b65c00 to 0x1b71e00, (0xc200 bytes) 2024-09-19T21:35:53.634 INFO:teuthology.orchestra.run.smithi122.stdout:3842 write 0x3c6c200 thru 0x3c6d9e6 (0x17e7 bytes) 2024-09-19T21:35:53.635 INFO:teuthology.orchestra.run.smithi122.stdout:3845 punch from 0x7c8b000 to 0x7c8d200, (0x2200 bytes) 2024-09-19T21:35:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:3848 punch from 0x7104c00 to 0x7112e00, (0xe200 bytes) 2024-09-19T21:35:53.636 INFO:teuthology.orchestra.run.smithi122.stdout:3849 read 0x92c4c00 thru 0x92c7983 (0x2d84 bytes) 2024-09-19T21:35:53.639 INFO:teuthology.orchestra.run.smithi122.stdout:3850 write 0xcaa6200 thru 0xcab0e33 (0xac34 bytes) 2024-09-19T21:35:53.639 INFO:teuthology.orchestra.run.smithi122.stdout:3853 punch from 0x98a0e00 to 0x98abc00, (0xae00 bytes) 2024-09-19T21:35:53.640 INFO:teuthology.orchestra.run.smithi122.stdout:3855 write 0x435c00 thru 0x43dcf2 (0x80f3 bytes) 2024-09-19T21:35:53.662 INFO:teuthology.orchestra.run.smithi122.stdout:3857 trunc from 0xdfc3e00 to 0x495be00 2024-09-19T21:35:53.700 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:53.692+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:53.710 INFO:teuthology.orchestra.run.smithi122.stdout:3858 punch from 0x227fa00 to 0x228a800, (0xae00 bytes) 2024-09-19T21:35:53.712 INFO:teuthology.orchestra.run.smithi122.stdout:3859 punch from 0x18f0600 to 0x18fce00, (0xc800 bytes) 2024-09-19T21:35:53.713 INFO:teuthology.orchestra.run.smithi122.stdout:3860 read 0x3b7b000 thru 0x3b826fc (0x76fd bytes) 2024-09-19T21:35:53.714 INFO:teuthology.orchestra.run.smithi122.stdout:3862 write 0x3dfa000 thru 0x3e02d66 (0x8d67 bytes) 2024-09-19T21:35:53.715 INFO:teuthology.orchestra.run.smithi122.stdout:3863 write 0x22f5800 thru 0x23011fd (0xb9fe bytes) 2024-09-19T21:35:53.717 INFO:teuthology.orchestra.run.smithi122.stdout:3865 punch from 0x180f400 to 0x1819000, (0x9c00 bytes) 2024-09-19T21:35:53.718 INFO:teuthology.orchestra.run.smithi122.stdout:3866 punch from 0x3345000 to 0x3346000, (0x1000 bytes) 2024-09-19T21:35:53.719 INFO:teuthology.orchestra.run.smithi122.stdout:3867 write 0x2a1e00 thru 0x2aea59 (0xcc5a bytes) 2024-09-19T21:35:53.723 INFO:teuthology.orchestra.run.smithi122.stdout:3868 punch from 0x3ab6800 to 0x3abe800, (0x8000 bytes) 2024-09-19T21:35:53.724 INFO:teuthology.orchestra.run.smithi122.stdout:3869 read 0x4711c00 thru 0x471aea9 (0x92aa bytes) 2024-09-19T21:35:53.726 INFO:teuthology.orchestra.run.smithi122.stdout:3870 read 0x391d800 thru 0x392ae4b (0xd64c bytes) 2024-09-19T21:35:53.729 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.729 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:53.750 INFO:teuthology.orchestra.run.smithi122.stdout:3872 write 0x52f6200 thru 0x52fb6fe (0x54ff bytes) 2024-09-19T21:35:53.755 INFO:teuthology.orchestra.run.smithi122.stdout:3876 punch from 0x2fcf600 to 0x2fda600, (0xb000 bytes) 2024-09-19T21:35:53.755 INFO:teuthology.orchestra.run.smithi122.stdout:3879 punch from 0x2231e00 to 0x2235600, (0x3800 bytes) 2024-09-19T21:35:53.765 INFO:teuthology.orchestra.run.smithi122.stdout:3880 trunc from 0x52fb800 to 0x8a38e00 2024-09-19T21:35:53.788 INFO:teuthology.orchestra.run.smithi122.stdout:3881 write 0x78b5200 thru 0x78c43bb (0xf1bc bytes) 2024-09-19T21:35:53.790 INFO:teuthology.orchestra.run.smithi122.stdout:3882 punch from 0x84da600 to 0x84e3000, (0x8a00 bytes) 2024-09-19T21:35:53.791 INFO:teuthology.orchestra.run.smithi122.stdout:3883 read 0x3b24800 thru 0x3b26741 (0x1f42 bytes) 2024-09-19T21:35:53.792 INFO:teuthology.orchestra.run.smithi122.stdout:3889 read 0x48a8c00 thru 0x48b4ea8 (0xc2a9 bytes) 2024-09-19T21:35:53.793 INFO:teuthology.orchestra.run.smithi122.stdout:3890 read 0x7486400 thru 0x7495f67 (0xfb68 bytes) 2024-09-19T21:35:53.794 INFO:teuthology.orchestra.run.smithi122.stdout:3891 read 0x42e9e00 thru 0x42e9f6e (0x16f bytes) 2024-09-19T21:35:53.797 INFO:teuthology.orchestra.run.smithi122.stdout:3892 read 0x153b000 thru 0x153fa77 (0x4a78 bytes) 2024-09-19T21:35:53.800 INFO:teuthology.orchestra.run.smithi122.stdout:3894 trunc from 0x8a38e00 to 0x7cdaa00 2024-09-19T21:35:53.801 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:53.794+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:53.822 INFO:teuthology.orchestra.run.smithi122.stdout:3896 read 0x603c600 thru 0x6041c8d (0x568e bytes) 2024-09-19T21:35:53.825 INFO:teuthology.orchestra.run.smithi122.stdout:3897 read 0x26cf800 thru 0x26d9506 (0x9d07 bytes) 2024-09-19T21:35:53.828 INFO:teuthology.orchestra.run.smithi122.stdout:3898 trunc from 0x7cdaa00 to 0x85b8a00 2024-09-19T21:35:53.849 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.849 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.852 INFO:teuthology.orchestra.run.smithi122.stdout:3901 read 0x3ec4600 thru 0x3ece724 (0xa125 bytes) 2024-09-19T21:35:53.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:53.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:53.872 INFO:teuthology.orchestra.run.smithi122.stdout:3903 trunc from 0x85b8a00 to 0xfcac00 2024-09-19T21:35:53.907 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:53.902+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:53.934 INFO:teuthology.orchestra.run.smithi122.stdout:3904 read 0xbb6a00 thru 0xbbd6a1 (0x6ca2 bytes) 2024-09-19T21:35:53.972 INFO:teuthology.orchestra.run.smithi122.stdout:3907 write 0x8a9c000 thru 0x8aa5b55 (0x9b56 bytes) 2024-09-19T21:35:53.977 INFO:teuthology.orchestra.run.smithi122.stdout:3910 trunc from 0x8aa5c00 to 0xa5a4000 2024-09-19T21:35:53.983 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:53.983 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:53.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:54.002 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:53.996+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:54.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:54.006 INFO:teuthology.orchestra.run.smithi122.stdout:3911 trunc from 0xa5a4000 to 0x6d1b400 2024-09-19T21:35:54.029 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:54.032 INFO:teuthology.orchestra.run.smithi122.stdout:3912 punch from 0x5fbbe00 to 0x5fbe600, (0x2800 bytes) 2024-09-19T21:35:54.033 INFO:teuthology.orchestra.run.smithi122.stdout:3916 punch from 0x4715e00 to 0x471c200, (0x6400 bytes) 2024-09-19T21:35:54.034 INFO:teuthology.orchestra.run.smithi122.stdout:3917 read 0x2661a00 thru 0x2665abb (0x40bc bytes) 2024-09-19T21:35:54.034 INFO:teuthology.orchestra.run.smithi122.stdout:3918 punch from 0x86e600 to 0x87d000, (0xea00 bytes) 2024-09-19T21:35:54.034 INFO:teuthology.orchestra.run.smithi122.stdout:3919 read 0xa06c00 thru 0xa0f361 (0x8762 bytes) 2024-09-19T21:35:54.035 INFO:teuthology.orchestra.run.smithi122.stdout:3920 punch from 0x5a9d000 to 0x5aa4800, (0x7800 bytes) 2024-09-19T21:35:54.036 INFO:teuthology.orchestra.run.smithi122.stdout:3921 read 0x3f4fa00 thru 0x3f5ed08 (0xf309 bytes) 2024-09-19T21:35:54.036 INFO:teuthology.orchestra.run.smithi122.stdout:3924 write 0x43d8600 thru 0x43dd75c (0x515d bytes) 2024-09-19T21:35:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:3926 trunc from 0x6d1b400 to 0x389f800 2024-09-19T21:35:54.049 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:54.072 INFO:teuthology.orchestra.run.smithi122.stdout:3927 punch from 0x4f2400 to 0x4f7200, (0x4e00 bytes) 2024-09-19T21:35:54.074 INFO:teuthology.orchestra.run.smithi122.stdout:3929 punch from 0x112e000 to 0x112ec00, (0xc00 bytes) 2024-09-19T21:35:54.075 INFO:teuthology.orchestra.run.smithi122.stdout:3931 punch from 0x1507800 to 0x150dc00, (0x6400 bytes) 2024-09-19T21:35:54.076 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:54.099 INFO:teuthology.orchestra.run.smithi122.stdout:3932 write 0x5482600 thru 0x5486499 (0x3e9a bytes) 2024-09-19T21:35:54.100 INFO:teuthology.orchestra.run.smithi122.stdout:3933 read 0x2cab600 thru 0x2cb7ed8 (0xc8d9 bytes) 2024-09-19T21:35:54.103 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:54.096+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:54.104 INFO:teuthology.orchestra.run.smithi122.stdout:3937 write 0x316e800 thru 0x317c0d8 (0xd8d9 bytes) 2024-09-19T21:35:54.104 INFO:teuthology.orchestra.run.smithi122.stdout:3939 punch from 0x176e00 to 0x182c00, (0xbe00 bytes) 2024-09-19T21:35:54.107 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:54.108 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.109 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:54.109 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:54.109 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:54.109 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.111 INFO:teuthology.orchestra.run.smithi122.stdout:3940 trunc from 0x5486600 to 0x75b7600 2024-09-19T21:35:54.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:54.130 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:54.134 INFO:teuthology.orchestra.run.smithi122.stdout:3941 punch from 0x59ebc00 to 0x59ed600, (0x1a00 bytes) 2024-09-19T21:35:54.136 INFO:teuthology.orchestra.run.smithi122.stdout:3946 write 0x3ed8600 thru 0x3edef5b (0x695c bytes) 2024-09-19T21:35:54.161 INFO:teuthology.orchestra.run.smithi122.stdout:3948 write 0x8b6c600 thru 0x8b7122d (0x4c2e bytes) 2024-09-19T21:35:54.166 INFO:teuthology.orchestra.run.smithi122.stdout:3950 punch from 0x60dde00 to 0x60e6c00, (0x8e00 bytes) 2024-09-19T21:35:54.167 INFO:teuthology.orchestra.run.smithi122.stdout:3951 read 0x1f00800 thru 0x1f0cbef (0xc3f0 bytes) 2024-09-19T21:35:54.171 INFO:teuthology.orchestra.run.smithi122.stdout:3953 read 0x6b8a00 thru 0x6c2130 (0x9731 bytes) 2024-09-19T21:35:54.173 INFO:teuthology.orchestra.run.smithi122.stdout:3955 write 0x22c0600 thru 0x22cf9de (0xf3df bytes) 2024-09-19T21:35:54.174 INFO:teuthology.orchestra.run.smithi122.stdout:3956 read 0x27b2c00 thru 0x27be044 (0xb445 bytes) 2024-09-19T21:35:54.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:35:54.209 INFO:teuthology.orchestra.run.smithi122.stdout:3959 write 0xb46c400 thru 0xb4761ab (0x9dac bytes) 2024-09-19T21:35:54.226 INFO:teuthology.orchestra.run.smithi122.stdout:3960 trunc from 0xb476200 to 0x4ca5e00 2024-09-19T21:35:54.231 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:54.226+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.232 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.871833926000001, 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:54.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.871821617999998, 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:54.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.839090379000002, 2024-09-19T21:35:54.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:54.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609332685999998, 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:54.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609309250999999, 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:54.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:54.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609287522999999, 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:54.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609266791, 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:54.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:54.263 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609232355, 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:54.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:54.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.609189558000001, 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.268 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:54.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603964048000002, 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:54.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603941750000001, 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:54.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603910894999999, 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:54.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:54.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603868186, 2024-09-19T21:35:54.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.290 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603844199000001, 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:54.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603812795, 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.295 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:54.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603787628999999, 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.299 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:54.303 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603742551, 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:54.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.307 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603715807, 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:54.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603702514999998, 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:54.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.603685595999998, 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.316 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.319 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.320 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.321 INFO:teuthology.orchestra.run.smithi122.stdout:3961 write 0xa506200 thru 0xa512798 (0xc599 bytes) 2024-09-19T21:35:54.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:54.322 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:54.314+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:54.348 INFO:teuthology.orchestra.run.smithi122.stdout:3962 write 0xbb41e00 thru 0xbb487bd (0x69be bytes) 2024-09-19T21:35:54.349 INFO:teuthology.orchestra.run.smithi122.stdout:3963 read 0xa66f000 thru 0xa6705d7 (0x15d8 bytes) 2024-09-19T21:35:54.384 INFO:teuthology.orchestra.run.smithi122.stdout:3964 write 0xdd8b200 thru 0xdd91581 (0x6382 bytes) 2024-09-19T21:35:54.386 INFO:teuthology.orchestra.run.smithi122.stdout:3965 write 0xb31b200 thru 0xb3257e4 (0xa5e5 bytes) 2024-09-19T21:35:54.411 INFO:teuthology.orchestra.run.smithi122.stdout:3966 trunc from 0xdd91600 to 0x12b4600 2024-09-19T21:35:54.422 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:54.416+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:54.442 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.442 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:54.442 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:54.442 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:54.499 INFO:teuthology.orchestra.run.smithi122.stdout:3971 punch from 0xdce400 to 0xdd9400, (0xb000 bytes) 2024-09-19T21:35:54.523 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:54.518+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:54.535 INFO:teuthology.orchestra.run.smithi122.stdout:3972 write 0x7bfba00 thru 0x7bfcb0a (0x110b bytes) 2024-09-19T21:35:54.554 INFO:teuthology.orchestra.run.smithi122.stdout:3973 trunc from 0x7bfcc00 to 0x95a400 2024-09-19T21:35:54.566 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.566 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:54.567 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:54.567 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:54.567 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:54.624 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:54.618+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:54.643 INFO:teuthology.orchestra.run.smithi122.stdout:3975 read 0x476400 thru 0x478d75 (0x2976 bytes) 2024-09-19T21:35:54.670 INFO:teuthology.orchestra.run.smithi122.stdout:3976 trunc from 0x95a400 to 0xc6f1800 2024-09-19T21:35:54.689 INFO:teuthology.orchestra.run.smithi122.stdout:3978 punch from 0x8dc4c00 to 0x8dca400, (0x5800 bytes) 2024-09-19T21:35:54.690 INFO:teuthology.orchestra.run.smithi122.stdout:3979 write 0x7fae600 thru 0x7fb4e4f (0x6850 bytes) 2024-09-19T21:35:54.690 INFO:teuthology.orchestra.run.smithi122.stdout:3980 read 0x26e7200 thru 0x26ed04d (0x5e4e bytes) 2024-09-19T21:35:54.691 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.691 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 98.755734660000002, 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:54.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:54.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:54.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.698 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436167329, 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:35:54.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.703 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436155563, 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:35:54.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.708 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436142669999999, 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436130646999999, 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:35:54.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:54.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:35:54.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:35:54.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436113002999999, 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:35:54.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436097207, 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:35:54.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436085356, 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.436016511999998, 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:54.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:35:54.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.435988372000001, 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.435965242999998, 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:54.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:35:54.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.421188622999999, 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:54.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:35:54.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.413852307999999, 2024-09-19T21:35:54.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:35:54.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.413831418000001, 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:35:54.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.406755877999998, 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:54.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.406735334, 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:35:54.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.226381263, 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:35:54.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.775 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.226370836000001, 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:35:54.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.226360407000001, 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:54.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.226350344, 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:54.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:35:54.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:54.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:54.789 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.791 INFO:teuthology.orchestra.run.smithi122.stdout:3981 write 0xd727400 thru 0xd731c2c (0xa82d bytes) 2024-09-19T21:35:54.791 INFO:teuthology.orchestra.run.smithi122.stdout:3982 read 0x5910600 thru 0x591983a (0x923b bytes) 2024-09-19T21:35:54.791 INFO:teuthology.orchestra.run.smithi122.stdout:3984 trunc from 0xd731e00 to 0xc646a00 2024-09-19T21:35:54.791 INFO:teuthology.orchestra.run.smithi122.stdout:3985 read 0x79bbe00 thru 0x79be117 (0x2318 bytes) 2024-09-19T21:35:54.791 INFO:teuthology.orchestra.run.smithi122.stdout:3988 trunc from 0xc646a00 to 0x9841c00 2024-09-19T21:35:54.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:54.792 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:54.782+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:54.799 INFO:teuthology.orchestra.run.smithi122.stdout:3994 read 0x69db800 thru 0x69e1117 (0x5918 bytes) 2024-09-19T21:35:54.803 INFO:teuthology.orchestra.run.smithi122.stdout:3995 read 0x6483600 thru 0x648ef90 (0xb991 bytes) 2024-09-19T21:35:54.803 INFO:teuthology.orchestra.run.smithi122.stdout:3996 write 0x212ba00 thru 0x2130b25 (0x5126 bytes) 2024-09-19T21:35:54.803 INFO:teuthology.orchestra.run.smithi122.stdout:3997 write 0x4062600 thru 0x4068995 (0x6396 bytes) 2024-09-19T21:35:54.838 INFO:teuthology.orchestra.run.smithi122.stdout:4001 write 0x9e47600 thru 0x9e4a559 (0x2f5a bytes) 2024-09-19T21:35:54.843 INFO:teuthology.orchestra.run.smithi122.stdout:4002 trunc from 0x9e4a600 to 0x8d86a00 2024-09-19T21:35:54.866 INFO:teuthology.orchestra.run.smithi122.stdout:4003 read 0x2190e00 thru 0x219c028 (0xb229 bytes) 2024-09-19T21:35:54.869 INFO:teuthology.orchestra.run.smithi122.stdout:4008 punch from 0x4581800 to 0x4584a00, (0x3200 bytes) 2024-09-19T21:35:54.869 INFO:teuthology.orchestra.run.smithi122.stdout:4009 read 0x4a0e800 thru 0x4a18119 (0x991a bytes) 2024-09-19T21:35:54.884 INFO:teuthology.orchestra.run.smithi122.stdout:4010 trunc from 0x8d86a00 to 0x2f47e00 2024-09-19T21:35:54.891 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:54.886+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:54.918 INFO:teuthology.orchestra.run.smithi122.stdout:4011 read 0x22bf400 thru 0x22c8932 (0x9533 bytes) 2024-09-19T21:35:54.953 INFO:teuthology.orchestra.run.smithi122.stdout:4012 write 0x8c40600 thru 0x8c4fa9b (0xf49c bytes) 2024-09-19T21:35:54.957 INFO:teuthology.orchestra.run.smithi122.stdout:4013 trunc from 0x8c4fc00 to 0x7f25c00 2024-09-19T21:35:54.958 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:54.959 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:54.959 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:54.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:54.991 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:54.984+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:54.997 INFO:teuthology.orchestra.run.smithi122.stdout:4015 trunc from 0x7f25c00 to 0x401c00 2024-09-19T21:35:55.082 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:55.082 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:55.082 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:55.082 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:55.083 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:55.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:55.093 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:55.086+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:55.105 INFO:teuthology.orchestra.run.smithi122.stdout:4017 trunc from 0x401c00 to 0xcfe3600 2024-09-19T21:35:55.126 INFO:teuthology.orchestra.run.smithi122.stdout:4018 write 0xc60f200 thru 0xc617335 (0x8136 bytes) 2024-09-19T21:35:55.129 INFO:teuthology.orchestra.run.smithi122.stdout:4023 clone 25 order 25 su 8388608 sc 11 2024-09-19T21:35:55.191 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:35:55.193 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:55.188+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:55.206 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:55.206 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:55.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:55.206 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.474999121000003, 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:35:55.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:35:55.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8730438770000006, 2024-09-19T21:35:55.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:35:55.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8726315299999996, 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:35:55.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8726146280000009, 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:35:55.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8725978950000002, 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8725168429999997, 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:35:55.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:35:55.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8725001129999992, 2024-09-19T21:35:55.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:35:55.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8724837159999996, 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:35:55.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:35:55.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8724672830000006, 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.244 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:55.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:35:55.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.248 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8724517269999996, 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:35:55.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:35:55.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8723376579999993, 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.253 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:35:55.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8723206949999991, 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.258 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:35:55.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:35:55.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.872304711, 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:35:55.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.872287536, 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:35:55.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:35:55.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8722703379999999, 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:35:55.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.275 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8722535069999999, 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:35:55.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8722369860000008, 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:35:55.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8722143819999992, 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:35:55.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8721979940000004, 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:35:55.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.293 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 8.8720891129999995, 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:55.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:35:55.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:55.297 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:55.298 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:55.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:55.299 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:55.292+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:55.348 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone24 from 0xcfe3600 (overlap 0x30800) to 0x18400 2024-09-19T21:35:55.400 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:55.394+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:55.431 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:55.431 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:55.431 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:55.431 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:55.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:55.500 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:55.494+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:55.553 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:55.553 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:55.553 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:55.553 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:55.553 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:55.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:55.601 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:55.594+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:55.677 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:55.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419833124999997, 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.678 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.682 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419819586000003, 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:35:55.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:35:55.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419793845999997, 2024-09-19T21:35:55.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419779632000001, 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:35:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419758715, 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:35:55.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419743339, 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.702 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:35:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419728411000001, 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:35:55.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:35:55.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.711 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419675791000003, 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:35:55.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:35:55.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:35:55.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419626213000001, 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:35:55.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419608998000001, 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:35:55.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419589860000002, 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:35:55.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419575012000003, 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:55.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:35:55.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:35:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.419552019000001, 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.735 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:35:55.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:35:55.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.418819726999999, 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:35:55.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:35:55.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.747 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.418769417, 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:35:55.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.418707105000003, 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:35:55.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:35:55.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.417376472000001, 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:35:55.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.417240894000003, 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:35:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.417164135, 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:35:55.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:35:55.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:35:55.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:55.772 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.417105411000001, 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.773 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:55.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:35:55.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:35:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:55.777 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:55.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:55.779 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:55.772+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:55.879 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:55.874+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:55.898 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:55.898 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:55.898 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:55.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:55.980 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:55.972+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:56.016 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:56.016 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:56.017 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:56.017 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:56.017 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:56.026 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:56.073 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-19T21:35:56.080 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:56.074+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.940952278000001, 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:56.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:35:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:35:56.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.940372366999998, 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:35:56.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:35:56.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:35:56.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:35:56.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:35:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.940309719999998, 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:35:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:35:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:35:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:35:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:35:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:35:56.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:35:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.9402434, 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:35:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:35:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:35:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:35:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:35:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:35:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:35:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:35:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:35:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:35:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:35:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:35:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939539721999999, 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:35:56.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:35:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:35:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:35:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:35:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:35:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:35:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:35:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:35:56.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:35:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:35:56.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.196 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939466899999999, 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:35:56.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.202 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939453162, 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:35:56.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939435473, 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:35:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939385087000002, 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:35:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:35:56.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939358198000001, 2024-09-19T21:35:56.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:35:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939317622000001, 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:35:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:35:56.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939296007999999, 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:56.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:35:56.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.939276808999999, 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:35:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:35:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.450559315, 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:56.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.450292038000001, 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:35:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.449959206999999, 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:35:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.449412063, 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.250 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:35:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:56.254 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.449335724999999, 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:35:56.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.449003448999999, 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:35:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:35:56.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.448700522999999, 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:56.263 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:56.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:56.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:56.267 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:56.268 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:56.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:35:56.270 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:56.262+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:56.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4024 read 0xc3fa600 thru 0xc3fb19a (0xb9b bytes) 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4026 trunc from 0xcfe3600 to 0xd1f6000 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4029 write 0x8951400 thru 0x895d55f (0xc160 bytes) 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4030 write 0x5398200 thru 0x53a1025 (0x8e26 bytes) 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4031 read 0x7509600 thru 0x7516139 (0xcb3a bytes) 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4032 write 0x69a7e00 thru 0x69aad12 (0x2f13 bytes) 2024-09-19T21:35:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:4034 write 0xa082e00 thru 0xa08d7a6 (0xa9a7 bytes) 2024-09-19T21:35:56.273 INFO:teuthology.orchestra.run.smithi122.stdout:4037 write 0x2aaee00 thru 0x2ab7e2c (0x902d bytes) 2024-09-19T21:35:56.273 INFO:teuthology.orchestra.run.smithi122.stdout:4038 punch from 0x61fcc00 to 0x61fee00, (0x2200 bytes) 2024-09-19T21:35:56.273 INFO:teuthology.orchestra.run.smithi122.stdout:4039 punch from 0x3ed5a00 to 0x3eda400, (0x4a00 bytes) 2024-09-19T21:35:56.273 INFO:teuthology.orchestra.run.smithi122.stdout:4040 punch from 0x5d68c00 to 0x5d6bc00, (0x3000 bytes) 2024-09-19T21:35:56.274 INFO:teuthology.orchestra.run.smithi122.stdout:4041 write 0xd355a00 thru 0xd3649aa (0xefab bytes) 2024-09-19T21:35:56.275 INFO:teuthology.orchestra.run.smithi122.stdout:4042 read 0x7597c00 thru 0x759e42f (0x6830 bytes) 2024-09-19T21:35:56.275 INFO:teuthology.orchestra.run.smithi122.stdout:4044 read 0x9f00800 thru 0x9f0f667 (0xee68 bytes) 2024-09-19T21:35:56.277 INFO:teuthology.orchestra.run.smithi122.stdout:4046 write 0xcf88200 thru 0xcf9122a (0x902b bytes) 2024-09-19T21:35:56.277 INFO:teuthology.orchestra.run.smithi122.stdout:4047 clone 26 order 19 su 262144 sc 12 2024-09-19T21:35:56.346 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:35:56.347 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:35:56.347 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-19T21:35:56.347 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:35:56.347 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:35:56.347 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-19T21:35:56.370 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:56.364+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:56.388 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:56.388 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:56.388 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:35:56.388 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:56.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:56.414 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:56.434 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:56.454 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:56.458 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:56.458 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "" 2024-09-19T21:35:56.458 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:56.471 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:56.464+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:56.474 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:56.497 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:56.509 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:56.510 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:56.510 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:56.510 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:56.510 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:56.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:56.521 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:56.572 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:56.566+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:56.626 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:56.626 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:56.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:56.683 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:56.678+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:56.754 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:56.754 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:56.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:56.774 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:56.766+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:56.867 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:56.867 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:56.876 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:56.870+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:56.877 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:56.977 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:56.972+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:57.030 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.030 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:57.077 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:57.070+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:57.144 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.144 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.154 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:57.178 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:57.172+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:57.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:35:57.264 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.264 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:57.279 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:35:57.274+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:35:57.367 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone25 from 0xd364a00 (overlap 0xcfe3600) to 0x0 2024-09-19T21:35:57.380 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:57.372+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:57.385 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.385 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.396 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:35:57.481 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:57.474+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:57.551 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.551 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.561 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:35:57.611 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:57.606+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:57.671 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.671 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:35:57.683 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:57.678+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:57.784 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:57.776+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:57.786 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.786 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:35:57.884 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:57.878+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:57.914 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:57.915 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:57.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:35:57.984 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:35:57.980+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:35:58.037 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.038 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.048 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:35:58.086 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:58.080+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:58.147 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-19T21:35:58.160 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.161 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:35:58.186 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:58.178+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:58.196 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:35:58.256 INFO:teuthology.orchestra.run.smithi122.stdout:4048 read 0xaf51200 thru 0xaf53cde (0x2adf bytes) 2024-09-19T21:35:58.262 INFO:teuthology.orchestra.run.smithi122.stdout:4049 write 0x3f44000 thru 0x3f45bb2 (0x1bb3 bytes) 2024-09-19T21:35:58.264 INFO:teuthology.orchestra.run.smithi122.stdout:4050 write 0x8db9000 thru 0x8dbf0ff (0x6100 bytes) 2024-09-19T21:35:58.269 INFO:teuthology.orchestra.run.smithi122.stdout:4051 write 0x9169000 thru 0x9174d6e (0xbd6f bytes) 2024-09-19T21:35:58.271 INFO:teuthology.orchestra.run.smithi122.stdout:4052 read 0x5f2e600 thru 0x5f3bfac (0xd9ad bytes) 2024-09-19T21:35:58.273 INFO:teuthology.orchestra.run.smithi122.stdout:4055 write 0x405de00 thru 0x40633e3 (0x55e4 bytes) 2024-09-19T21:35:58.274 INFO:teuthology.orchestra.run.smithi122.stdout:4056 read 0xa325800 thru 0xa32b6ed (0x5eee bytes) 2024-09-19T21:35:58.276 INFO:teuthology.orchestra.run.smithi122.stdout:4060 write 0x83e5200 thru 0x83efd78 (0xab79 bytes) 2024-09-19T21:35:58.278 INFO:teuthology.orchestra.run.smithi122.stdout:4061 punch from 0x4eef200 to 0x4efe200, (0xf000 bytes) 2024-09-19T21:35:58.287 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:58.282+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:58.317 INFO:teuthology.orchestra.run.smithi122.stdout:4063 trunc from 0xd364a00 to 0xd735200 2024-09-19T21:35:58.328 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.328 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:35:58.341 INFO:teuthology.orchestra.run.smithi122.stdout:4064 punch from 0x94fc200 to 0x950a400, (0xe200 bytes) 2024-09-19T21:35:58.342 INFO:teuthology.orchestra.run.smithi122.stdout:4065 read 0x7c09000 thru 0x7c1724b (0xe24c bytes) 2024-09-19T21:35:58.374 INFO:teuthology.orchestra.run.smithi122.stdout:4068 trunc from 0xd735200 to 0x97b2000 2024-09-19T21:35:58.388 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:58.380+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:58.441 INFO:teuthology.orchestra.run.smithi122.stdout:4076 write 0xa3b6c00 thru 0xa3bfd26 (0x9127 bytes) 2024-09-19T21:35:58.442 INFO:teuthology.orchestra.run.smithi122.stdout:4077 write 0x74f800 thru 0x750807 (0x1008 bytes) 2024-09-19T21:35:58.446 INFO:teuthology.orchestra.run.smithi122.stdout:4082 write 0x104d000 thru 0x105b439 (0xe43a bytes) 2024-09-19T21:35:58.447 INFO:teuthology.orchestra.run.smithi122.stdout:4083 read 0x69ff200 thru 0x6a0c949 (0xd74a bytes) 2024-09-19T21:35:58.447 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.448 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.448 INFO:teuthology.orchestra.run.smithi122.stdout:4086 punch from 0x6267c00 to 0x6272400, (0xa800 bytes) 2024-09-19T21:35:58.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:35:58.474 INFO:teuthology.orchestra.run.smithi122.stdout:4090 trunc from 0xa3bfe00 to 0x6cdde00 2024-09-19T21:35:58.488 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:58.482+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:58.521 INFO:teuthology.orchestra.run.smithi122.stdout:4092 write 0xa25c00 thru 0xa2cdb9 (0x71ba bytes) 2024-09-19T21:35:58.524 INFO:teuthology.orchestra.run.smithi122.stdout:4095 read 0x5444600 thru 0x544cc56 (0x8657 bytes) 2024-09-19T21:35:58.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:35:58.559 INFO:teuthology.orchestra.run.smithi122.stdout:4097 write 0x7afec00 thru 0x7b0011a (0x151b bytes) 2024-09-19T21:35:58.571 INFO:teuthology.orchestra.run.smithi122.stdout:4098 trunc from 0x7b00200 to 0xc03d200 2024-09-19T21:35:58.589 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:58.582+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:58.597 INFO:teuthology.orchestra.run.smithi122.stdout:4106 read 0x490ae00 thru 0x4919398 (0xe599 bytes) 2024-09-19T21:35:58.599 INFO:teuthology.orchestra.run.smithi122.stdout:4107 punch from 0xbc0a200 to 0xbc12200, (0x8000 bytes) 2024-09-19T21:35:58.600 INFO:teuthology.orchestra.run.smithi122.stdout:4109 punch from 0x9793200 to 0x97a2400, (0xf200 bytes) 2024-09-19T21:35:58.600 INFO:teuthology.orchestra.run.smithi122.stdout:4110 clone 27 order 20 su 524288 sc 6 2024-09-19T21:35:58.612 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.612 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.622 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:35:58.684 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:35:58.691 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:58.686+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:58.708 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:35:58.727 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:35:58.727 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:35:58.730 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:35:58.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:35:58.752 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:35:58.773 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:35:58.791 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:58.786+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:58.796 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:35:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:35:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:35:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:35:58.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:35:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:35:58.799 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:35:58.847 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:58.847 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:35:58.847 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:35:58.847 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:35:58.847 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:58.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:35:58.892 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:58.886+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:58.964 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 63.211770043000001, 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:35:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:35:58.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:35:58.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:35:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584430647000001, 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:35:58.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584378181000002, 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:35:58.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:35:58.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584347634, 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:35:58.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:35:58.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584325036999999, 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:35:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:35:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584302585, 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:35:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584288289, 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:58.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:35:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584271661999999, 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:35:58.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:35:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:35:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584253025999999, 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:35:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.007 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584226360999999, 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:59.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.58421615, 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:35:59.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.584203664, 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:35:59.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580329074000002, 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:35:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580311555000002, 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:35:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580274000999999, 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:35:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:35:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580256158000001, 2024-09-19T21:35:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:35:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580243709000001, 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580233017000001, 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:35:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:35:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.580221439999999, 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:35:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:35:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 18.164249483999999, 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:35:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:35:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:35:59.057 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:35:59.058 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:35:59.059 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:59.050+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:59.159 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:59.156+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:59.177 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.177 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:59.177 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:35:59.177 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:59.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:35:59.260 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:35:59.254+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:35:59.299 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.299 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:59.299 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:59.299 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:59.299 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:59.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:35:59.361 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:59.354+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:59.402 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone26 from 0xc03d200 (overlap 0x6cdde00) to 0x0 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.075384621000001, 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:59.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.075372312999999, 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:35:59.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:35:59.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.042641073999999, 2024-09-19T21:35:59.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:35:59.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:35:59.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.812883380999999, 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:35:59.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:35:59.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:35:59.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.812859946, 2024-09-19T21:35:59.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:35:59.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.812838218, 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:35:59.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.812817486, 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.464 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:35:59.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:35:59.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.81278305, 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:35:59.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:35:59.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.812740253000001, 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.476 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807514742999999, 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807492445000001, 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:35:59.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:35:59.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:35:59.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807461589999999, 2024-09-19T21:35:59.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.489 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807418881, 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:35:59.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:35:59.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807394894000002, 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.497 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.80736349, 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:59.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:35:59.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807338324, 2024-09-19T21:35:59.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807293246, 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:35:59.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807266502000001, 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807253209999999, 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 26.807236290999999, 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:35:59.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.521 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.521 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.521 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:59.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:35:59.523 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:59.516+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:59.623 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:35:59.618+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:35:59.679 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:59.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5186.0:2264 3.d 3:b7c603b6:::rbd_object_map.11067d76f973:head [call lock.unlock in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:59.677634+0000", 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.001114103, 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011751979999999999, 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for sub ops", 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5186", 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3441894866", 2024-09-19T21:35:59.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2264 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677634+0000", 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677634+0000", 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677630+0000", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999967 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677655+0000", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4501000000000001e-05 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677703+0000", 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8025000000000002e-05 2024-09-19T21:35:59.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677721+0000", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8097e-05 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.677842+0000", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000121103 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.678008+0000", 2024-09-19T21:35:59.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016592500000000001 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:59.678312+0000", 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000303744 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:35:59.684 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:59.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:35:59.724 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:35:59.716+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:35:59.794 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.794 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:35:59.794 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:35:59.794 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:35:59.794 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:35:59.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:35:59.825 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:35:59.820+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:59.919 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 103.982816267, 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:59.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:35:59.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:35:59.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:35:59.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:59.926 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663248935999999, 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:35:59.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663237169999999, 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.935 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663224277000001, 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:59.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:35:59.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663212254000001, 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:59.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:35:59.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663194610000001, 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663178813999998, 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:35:59.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663166962999998, 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:35:59.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:35:59.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:35:59.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663098119000001, 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:35:59.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:35:59.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:35:59.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663069978999999, 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:35:59.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.663046850000001, 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:35:59.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:35:59.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.648270230000001, 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.640933915000002, 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.976 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:59.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:35:59.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.980 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.640913025, 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:59.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:35:59.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:35:59.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.633837485000001, 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:35:59.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:35:59.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.633816940999999, 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.989 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:35:59.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.453462869999999, 2024-09-19T21:35:59.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:35:59.997 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.453452443, 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:59.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:35:59.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:00.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.453442014, 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.005 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.453431950999999, 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.009 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.010 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.010 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.010 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:00.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:00.012 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:00.004+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:00.112 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:00.104+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:00.127 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:00.127 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:00.127 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:00.127 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:00.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:00.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:36:00.212 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:00.206+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:00.245 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:00.245 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:00.245 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:00.245 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:00.245 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:00.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:00.266 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-19T21:36:00.313 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:00.308+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:00.365 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.633851312999994, 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:00.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:00.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031896069, 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:00.372 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:00.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:00.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031483722000001, 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:36:00.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:36:00.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.381 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.03146682, 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:36:00.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031450087, 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:00.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031369035000001, 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:36:00.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031352305, 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:00.399 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031335908000001, 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:00.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031319475, 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:00.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:00.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031303919000001, 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:00.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:00.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:00.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:00.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031189850000001, 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:00.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:00.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031172887, 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:00.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:00.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031156902999999, 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.423 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:00.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:00.427 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031139727999999, 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:00.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:00.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031122529999999, 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:00.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031105698999999, 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.437 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:00.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:00.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:00.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:00.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031089178, 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:00.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:00.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:00.449 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031066574, 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:00.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:00.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.031050186, 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:00.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.030941305000001, 2024-09-19T21:36:00.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:00.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:00.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:00.462 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:00.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:00.465 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:00.458+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:00.466 INFO:teuthology.orchestra.run.smithi122.stdout:4111 read 0x82fba00 thru 0x8300f89 (0x558a bytes) 2024-09-19T21:36:00.466 INFO:teuthology.orchestra.run.smithi122.stdout:4113 read 0x6374e00 thru 0x6379ae3 (0x4ce4 bytes) 2024-09-19T21:36:00.466 INFO:teuthology.orchestra.run.smithi122.stdout:4116 read 0xaea6200 thru 0xaeab536 (0x5337 bytes) 2024-09-19T21:36:00.466 INFO:teuthology.orchestra.run.smithi122.stdout:4117 read 0x65bce00 thru 0x65c997f (0xcb80 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4118 read 0x1e09200 thru 0x1e0b707 (0x2508 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4119 read 0x506cc00 thru 0x506e469 (0x186a bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4120 read 0x2e73400 thru 0x2e77d72 (0x4973 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4123 write 0xd687e00 thru 0xd693e6e (0xc06f bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4126 write 0x29b3e00 thru 0x29bf95a (0xbb5b bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4127 write 0x7045a00 thru 0x70513ac (0xb9ad bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4129 read 0x25c4a00 thru 0x25cdf7c (0x957d bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4130 write 0xdb8a400 thru 0xdb9238e (0x7f8f bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4132 punch from 0x88a0c00 to 0x88a9e00, (0x9200 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4133 write 0x1942200 thru 0x1951828 (0xf629 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4134 write 0x843fe00 thru 0x84449bb (0x4bbc bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4136 write 0x2591200 thru 0x259b370 (0xa171 bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4137 read 0x715a00 thru 0x7210bd (0xb6be bytes) 2024-09-19T21:36:00.467 INFO:teuthology.orchestra.run.smithi122.stdout:4143 read 0x1ce9200 thru 0x1cf2238 (0x9039 bytes) 2024-09-19T21:36:00.486 INFO:teuthology.orchestra.run.smithi122.stdout:4145 trunc from 0xdb92400 to 0x8b8ba00 2024-09-19T21:36:00.529 INFO:teuthology.orchestra.run.smithi122.stdout:4147 write 0x6342400 thru 0x634c27d (0x9e7e bytes) 2024-09-19T21:36:00.532 INFO:teuthology.orchestra.run.smithi122.stdout:4148 punch from 0x5bf1800 to 0x5bffc00, (0xe400 bytes) 2024-09-19T21:36:00.533 INFO:teuthology.orchestra.run.smithi122.stdout:4149 punch from 0x6eb3200 to 0x6ebce00, (0x9c00 bytes) 2024-09-19T21:36:00.564 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:00.558+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:00.569 INFO:teuthology.orchestra.run.smithi122.stdout:4150 trunc from 0x8b8ba00 to 0x3ad5200 2024-09-19T21:36:00.585 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:00.586 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:00.586 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:00.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:00.605 INFO:teuthology.orchestra.run.smithi122.stdout:4151 read 0x582800 thru 0x58e35a (0xbb5b bytes) 2024-09-19T21:36:00.608 INFO:teuthology.orchestra.run.smithi122.stdout:4154 write 0x151cc00 thru 0x1525228 (0x8629 bytes) 2024-09-19T21:36:00.631 INFO:teuthology.orchestra.run.smithi122.stdout:4155 trunc from 0x3ad5200 to 0x10b3a00 2024-09-19T21:36:00.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:00.660+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:00.690 INFO:teuthology.orchestra.run.smithi122.stdout:4158 trunc from 0x10b3a00 to 0x59e9800 2024-09-19T21:36:00.701 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:00.701 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:00.701 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:00.701 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:00.701 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:00.711 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:00.713 INFO:teuthology.orchestra.run.smithi122.stdout:4159 read 0x267a600 thru 0x267eb8e (0x458f bytes) 2024-09-19T21:36:00.714 INFO:teuthology.orchestra.run.smithi122.stdout:4160 read 0x343a400 thru 0x343e93f (0x4540 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4162 read 0x4708200 thru 0x470ecef (0x6af0 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4164 read 0x43bda00 thru 0x43be1e7 (0x7e8 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4166 read 0x34a9800 thru 0x34ad360 (0x3b61 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4168 punch from 0x4849200 to 0x484d200, (0x4000 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4169 read 0x42d6600 thru 0x42d7121 (0xb22 bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4170 write 0x50bb000 thru 0x50bd479 (0x247a bytes) 2024-09-19T21:36:00.715 INFO:teuthology.orchestra.run.smithi122.stdout:4172 punch from 0x17be00 to 0x181800, (0x5a00 bytes) 2024-09-19T21:36:00.716 INFO:teuthology.orchestra.run.smithi122.stdout:4173 read 0x4363400 thru 0x4370298 (0xce99 bytes) 2024-09-19T21:36:00.716 INFO:teuthology.orchestra.run.smithi122.stdout:4174 read 0x5907400 thru 0x590f441 (0x8042 bytes) 2024-09-19T21:36:00.745 INFO:teuthology.orchestra.run.smithi122.stdout:4178 write 0xb3f6800 thru 0xb3f89e5 (0x21e6 bytes) 2024-09-19T21:36:00.766 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:00.760+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:00.771 INFO:teuthology.orchestra.run.smithi122.stdout:4179 write 0xc10c800 thru 0xc11992f (0xd130 bytes) 2024-09-19T21:36:00.776 INFO:teuthology.orchestra.run.smithi122.stdout:4182 trunc from 0xc119a00 to 0xd6db800 2024-09-19T21:36:00.803 INFO:teuthology.orchestra.run.smithi122.stdout:4184 punch from 0x539be00 to 0x53a1600, (0x5800 bytes) 2024-09-19T21:36:00.805 INFO:teuthology.orchestra.run.smithi122.stdout:4186 write 0x7377000 thru 0x737ff47 (0x8f48 bytes) 2024-09-19T21:36:00.805 INFO:teuthology.orchestra.run.smithi122.stdout:4187 write 0x4f47e00 thru 0x4f50274 (0x8475 bytes) 2024-09-19T21:36:00.805 INFO:teuthology.orchestra.run.smithi122.stdout:4190 clone 28 order 25 su 524288 sc 6 2024-09-19T21:36:00.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:00.828 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:00.828 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571505848999998, 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.829 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571492309999996, 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:00.838 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571466569999998, 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:00.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.842 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571452356000002, 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:00.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.847 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571431439000001, 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.848 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:00.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:00.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571416063000001, 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.857 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571401135000002, 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:00.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:00.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:00.862 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571348514999997, 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:00.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:00.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571298937000002, 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:00.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571281722000002, 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:00.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571262584000003, 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:00.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.884 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571247735999997, 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.571224743000002, 2024-09-19T21:36:00.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:00.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.570492451, 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:00.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.570442141000001, 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:00.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:00.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:00.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.570379828999997, 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:00.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.569049196000002, 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:00.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:00.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.568913618000003, 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.912 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:00.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.568836859000001, 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:00.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:00.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.568778135000002, 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:00.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:00.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:00.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:00.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:00.928 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:00.922+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:00.957 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:00.979 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:01.000 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:01.022 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:01.027 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:01.020+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:01.043 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:01.067 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:01.069 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:01.070 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.092 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:01.128 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:01.122+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:01.201 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.201 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:01.213 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:36:01.229 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:01.222+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:01.316 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.316 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:01.330 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:01.322+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:01.430 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:01.424+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:01.433 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone27 from 0xd6db800 (overlap 0x10b3a00) to 0x357200 2024-09-19T21:36:01.437 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.437 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:01.468 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:01.468 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:01.468 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:01.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-19T21:36:01.531 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:01.526+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:01.600 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.600 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:01.632 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:01.624+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:01.716 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.716 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:01.769 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:01.764+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:01.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-19T21:36:01.782+0000 7fba43fff700 0 --2- 172.21.15.12:0/3289569554 >> [v2:172.21.15.12:3301/0,v1:172.21.15.12:6790/0] conn(0x7fba4406d560 0x7fba441c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-19T21:36:01.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:01.828+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:01.859 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.859 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:01.933 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:01.928+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:01.979 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:01.980 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:01.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:02.016 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:36:02.010+0000 7eff847dd700 -1 mon.a@0(leader).osd e78 definitely_dead 0 2024-09-19T21:36:02.034 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:02.026+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:02.097 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.098 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:02.135 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:02.130+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:02.216 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.5. 2024-09-19T21:36:02.227 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.227 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.236 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:02.228+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:02.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:02.250 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-19T21:36:02.337 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:02.330+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:02.392 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.392 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:02.438 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:02.432+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.512 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:02.539 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:02.532+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:02.630 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.630 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.640 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:02.640 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:02.632+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:02.741 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:02.734+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:02.750 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.750 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:02.874 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:02.877 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:02.836+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:02.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:02.942 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:02.936+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:03.000 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:03.000 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:03.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:03.043 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:03.036+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:03.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:03.119 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:03.119 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:03.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:03.144 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:03.136+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:03.227 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:36:03.236 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:03.237 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:03.238 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:03.245 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:03.238+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:03.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:03.259 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:03.280 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:03.306 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:03.328 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:03.345 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:03.338+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:03.350 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:03.352 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:03.404 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:03.404 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:03.404 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:03.404 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:03.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:03.446 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:03.440+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:03.526 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:03.526 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:03.526 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:03.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:03.547 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:03.542+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:03.646 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:03.646 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.444528689999999, 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:03.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443948779000003, 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:03.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:03.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443886132000003, 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443819812000001, 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443116134, 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:03.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:03.672 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443043312, 2024-09-19T21:36:03.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443029574000001, 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.443011884999997, 2024-09-19T21:36:03.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:03.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.442961498999999, 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:03.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.442934610000002, 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:03.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:03.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:03.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.442894033999998, 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:03.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:03.703 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.44287242, 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:03.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.442853221, 2024-09-19T21:36:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:03.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.714 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.954135727000001, 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:03.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:03.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:03.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.722 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.953868450000002, 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.723 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:03.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.953535619, 2024-09-19T21:36:03.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.952988475000002, 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:03.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:03.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.952912136999998, 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:03.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.952579861, 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:03.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:03.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:03.748 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 21.952276935, 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:03.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:03.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:03.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:03.754 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:03.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:03.757 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:03.748+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:03.847 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:36:03.856 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:03.850+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:03.866 INFO:teuthology.orchestra.run.smithi122.stdout:4191 read 0xa27a200 thru 0xa28336b (0x916c bytes) 2024-09-19T21:36:03.867 INFO:teuthology.orchestra.run.smithi122.stdout:4192 punch from 0xbbf8200 to 0xbbf9000, (0xe00 bytes) 2024-09-19T21:36:03.884 INFO:teuthology.orchestra.run.smithi122.stdout:4193 read 0xb32fe00 thru 0xb33b5ca (0xb7cb bytes) 2024-09-19T21:36:03.884 INFO:teuthology.orchestra.run.smithi122.stdout:4196 read 0xa961e00 thru 0xa96ba81 (0x9c82 bytes) 2024-09-19T21:36:03.893 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:03.893 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:03.893 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:03.893 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:03.900 INFO:teuthology.orchestra.run.smithi122.stdout:4200 trunc from 0xd6db800 to 0xbd73000 2024-09-19T21:36:03.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:03.955 INFO:teuthology.orchestra.run.smithi122.stdout:4202 punch from 0x42f5200 to 0x4301a00, (0xc800 bytes) 2024-09-19T21:36:03.958 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:03.952+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:03.959 INFO:teuthology.orchestra.run.smithi122.stdout:4203 read 0x8965e00 thru 0x89697cb (0x39cc bytes) 2024-09-19T21:36:03.960 INFO:teuthology.orchestra.run.smithi122.stdout:4204 write 0xa3ae800 thru 0xa3b6a49 (0x824a bytes) 2024-09-19T21:36:03.960 INFO:teuthology.orchestra.run.smithi122.stdout:4205 read 0x9c8be00 thru 0x9c9512f (0x9330 bytes) 2024-09-19T21:36:04.010 INFO:teuthology.orchestra.run.smithi122.stdout:4209 trunc from 0xbd73000 to 0x50cec00 2024-09-19T21:36:04.029 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:04.030 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:04.030 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:04.030 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:04.030 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:04.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:04.058 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:04.054+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:04.080 INFO:teuthology.orchestra.run.smithi122.stdout:4211 write 0xb248400 thru 0xb2534a7 (0xb0a8 bytes) 2024-09-19T21:36:04.081 INFO:teuthology.orchestra.run.smithi122.stdout:4212 read 0x15de400 thru 0x15ee3df (0xffe0 bytes) 2024-09-19T21:36:04.082 INFO:teuthology.orchestra.run.smithi122.stdout:4217 read 0x5662e00 thru 0x566df8a (0xb18b bytes) 2024-09-19T21:36:04.083 INFO:teuthology.orchestra.run.smithi122.stdout:4220 write 0x6805a00 thru 0x680874b (0x2d4c bytes) 2024-09-19T21:36:04.084 INFO:teuthology.orchestra.run.smithi122.stdout:4221 read 0xaec0800 thru 0xaec2afd (0x22fe bytes) 2024-09-19T21:36:04.106 INFO:teuthology.orchestra.run.smithi122.stdout:4224 trunc from 0xb253600 to 0x4442600 2024-09-19T21:36:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 68.398771534999995, 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771432139000002, 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:04.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771379672999998, 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.163 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771349126, 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771326529, 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771304077, 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771289781, 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771273153999999, 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771254517999999, 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771227852999999, 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771217642, 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.771205156000001, 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767330565999998, 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767313047000002, 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767275493, 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767257650000001, 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767245201000001, 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767234509000001, 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:04.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:04.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.767222931999999, 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 23.351250975999999, 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:04.254 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:04.255 INFO:teuthology.orchestra.run.smithi122.stdout:4226 trunc from 0x4442600 to 0x8438000 2024-09-19T21:36:04.255 INFO:teuthology.orchestra.run.smithi122.stdout:4227 write 0xabe7600 thru 0xabf51d5 (0xdbd6 bytes) 2024-09-19T21:36:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:4228 write 0xd5c5400 thru 0xd5c845a (0x305b bytes) 2024-09-19T21:36:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:4229 write 0xa4b9c00 thru 0xa4c6675 (0xca76 bytes) 2024-09-19T21:36:04.256 INFO:teuthology.orchestra.run.smithi122.stdout:4230 trunc from 0xd5c8600 to 0xa8ee400 2024-09-19T21:36:04.256 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:04.257 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:04.250+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:04.281 INFO:teuthology.orchestra.run.smithi122.stdout:4231 read 0x699d600 thru 0x69a196b (0x436c bytes) 2024-09-19T21:36:04.283 INFO:teuthology.orchestra.run.smithi122.stdout:4233 punch from 0x8df0200 to 0x8dfbe00, (0xbc00 bytes) 2024-09-19T21:36:04.284 INFO:teuthology.orchestra.run.smithi122.stdout:4235 write 0x275b600 thru 0x2767062 (0xba63 bytes) 2024-09-19T21:36:04.285 INFO:teuthology.orchestra.run.smithi122.stdout:4236 punch from 0x911b000 to 0x912aa00, (0xfa00 bytes) 2024-09-19T21:36:04.286 INFO:teuthology.orchestra.run.smithi122.stdout:4240 read 0xc60000 thru 0xc64a11 (0x4a12 bytes) 2024-09-19T21:36:04.291 INFO:teuthology.orchestra.run.smithi122.stdout:4242 write 0x8777000 thru 0x8779ab9 (0x2aba bytes) 2024-09-19T21:36:04.293 INFO:teuthology.orchestra.run.smithi122.stdout:4245 write 0x1b28200 thru 0x1b331eb (0xafec bytes) 2024-09-19T21:36:04.295 INFO:teuthology.orchestra.run.smithi122.stdout:4246 read 0x5dea400 thru 0x5df75eb (0xd1ec bytes) 2024-09-19T21:36:04.297 INFO:teuthology.orchestra.run.smithi122.stdout:4248 write 0x50cf400 thru 0x50d6ce2 (0x78e3 bytes) 2024-09-19T21:36:04.298 INFO:teuthology.orchestra.run.smithi122.stdout:4250 read 0x46c3c00 thru 0x46d0308 (0xc709 bytes) 2024-09-19T21:36:04.299 INFO:teuthology.orchestra.run.smithi122.stdout:4252 read 0x3e56000 thru 0x3e5d561 (0x7562 bytes) 2024-09-19T21:36:04.300 INFO:teuthology.orchestra.run.smithi122.stdout:4253 clone 29 order 25 su 131072 sc 10 2024-09-19T21:36:04.356 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:04.350+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:04.378 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:04.378 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:04.378 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:04.378 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:04.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:04.456 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:04.450+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:04.504 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:04.504 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:04.504 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:04.504 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:04.505 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:04.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:04.558 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:04.552+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.271948780000002, 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:36:04.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.639 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.271936472, 2024-09-19T21:36:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:36:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:04.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:04.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:36:04.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.239205233, 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009447539999996, 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:36:04.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:36:04.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:04.656 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009424105000001, 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:04.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:04.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:04.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009402377000001, 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.662 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:04.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:04.666 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009381644999998, 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:04.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.671 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009347208999998, 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:04.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.009304411999999, 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.681 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.004078902000003, 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:04.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.004056603999999, 2024-09-19T21:36:04.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:04.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.004025749, 2024-09-19T21:36:04.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:04.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:04.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003983040000001, 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:04.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.700 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003959053000003, 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:04.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003927648999998, 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:04.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:04.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003902482999997, 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:04.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:04.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003857404999998, 2024-09-19T21:36:04.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:04.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:04.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.719 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003830661000002, 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:04.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.003817368999997, 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:04.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:04.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.730 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.00380045, 2024-09-19T21:36:04.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:04.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:04.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:04.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.738 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:04.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:04.739 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:04.735+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:04.840 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:04.835+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5231.0:966 3.10 3.c83748d0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:04.841082+0000", 2024-09-19T21:36:04.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.059477730999999999, 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.059496845999999999, 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5231", 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3576498036", 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 966 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841082+0000", 2024-09-19T21:36:04.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841082+0000", 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841080+0000", 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841086+0000", 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0920000000000003e-06 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841103+0000", 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6629e-05 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841109+0000", 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2489999999999998e-06 2024-09-19T21:36:04.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841189+0000", 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9940999999999999e-05 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.841242+0000", 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2556000000000003e-05 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDPGRecoveryDeleteReply(3.1f e82,80 3:fb594662:::rbd_data.1106c09d33d6.0000000000000010:head,73'290)", 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:04.900487+0000", 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.2347000000000004e-05, 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000132411, 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900487+0000", 2024-09-19T21:36:04.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900487+0000", 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900486+0000", 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:04.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900510+0000", 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3966999999999999e-05 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900532+0000", 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2801000000000001e-05 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:04.900548+0000", 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5965e-05 2024-09-19T21:36:04.908 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 2 2024-09-19T21:36:04.909 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:04.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:04.955 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:04.951+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:05.008 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:05.009 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:05.009 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:05.009 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:05.009 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:05.017 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:05.041 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:05.036+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.190307299, 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:05.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:05.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:05.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870739968000002, 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.133 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:05.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.137 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870728202000002, 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:05.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870715308999998, 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:05.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:05.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.147 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870703286000001, 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:05.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:05.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870685641999998, 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:05.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870669845999998, 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.162 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:05.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.166 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870657995000002, 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:05.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:05.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.171 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870589150999997, 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:05.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:05.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:05.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:05.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:05.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.178 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870561010999999, 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:05.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:05.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:05.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:05.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:05.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:05.305 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.870537882000001, 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:05.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:05.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.855761262000001, 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:05.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:05.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:05.315 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.848424946999998, 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:05.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:05.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.848404057000003, 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.841328517000001, 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:05.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:05.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.329 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.841307972999999, 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:05.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:05.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.334 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.660953902000003, 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.335 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:05.339 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.660943475000003, 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:05.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:05.344 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.660933045999997, 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:05.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.346 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:05.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:05.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.660922982999999, 2024-09-19T21:36:05.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:05.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:05.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:05.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:05.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.360 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:05.361 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:05.361 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:05.362 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:05.363 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:05.359+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:05.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:05.463 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:05.460+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:05.533 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:05.534 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:05.534 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:05.534 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:05.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:05.563 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:05.563 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:05.559+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:05.585 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:05.606 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:05.629 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:05.651 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:05.651 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:05.652 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:05.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:05.664 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:05.656+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:05.675 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:05.677 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:05.678 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:05.678 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:05.678 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:05.678 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:05.678 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:05.763 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:05.763 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:05.765 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:05.760+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:05.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:05.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:36:05.865 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:05.859+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:05.903 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:05.903 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:05.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:05.969 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:05.963+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:06.018 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.018 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:06.070 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:06.067+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:06.142 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.142 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:06.171 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:06.164+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:06.260 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.260 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:06.313 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:06.308+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:06.376 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:06.368+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:06.425 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.425 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:06.475 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:06.468+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:06.539 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.539 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:06.576 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:06.575+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:06.677 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:06.672+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:06.777 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:06.772+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:06.826 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.826 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:06.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:36:06.878 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:06.872+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:06.921 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone28 from 0xa8ee400 (overlap 0x4442600) to 0x0 2024-09-19T21:36:06.978 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:06.971+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:06.994 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:06.994 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.004 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:07.079 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:07.075+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:07.124 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:07.124 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:07.180 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:07.172+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:07.229 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:07.229 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:07.229 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-19T21:36:07.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-19T21:36:07.281 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:07.275+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:07.299 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:07.300 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.310 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:07.382 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:07.376+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:07.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-19T21:36:07.464 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:07.465 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:07.482 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:07.476+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:07.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-19T21:36:07.579 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:07.579 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.583 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:07.576+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:07.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:07.684 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:07.679+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:07.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:07.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-19T21:36:07.744 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:07.744 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:07.754 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:07.784 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:07.776+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:07.844 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:07.879 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:07.885 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:07.880+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:07.888 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:07.888 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:07.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 111.952543846, 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:07.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:07.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:07.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.895 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632976515000003, 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:07.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.900 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632964749000003, 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:07.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:07.905 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632951855999998, 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:07.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:07.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632939833000002, 2024-09-19T21:36:07.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:07.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:07.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632922188999999, 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:07.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:07.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632906392999999, 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:07.924 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632894542000003, 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632825697999998, 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:07.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:07.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.933 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632797558, 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.934 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:07.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.632774429000001, 2024-09-19T21:36:07.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.617997809000002, 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.610661493999999, 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.610640603999997, 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.603565064000001, 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.60354452, 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.423190449000003, 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.423180021999997, 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.423169592999997, 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.42315953, 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:07.985 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:07.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-19T21:36:07.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:07.988 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:07.988 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:07.990 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:07.983+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:08.087 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:08.080+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:08.089 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-19T21:36:08.152 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:08.152 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:08.152 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:08.152 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:08.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:08.188 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:08.183+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:08.207 INFO:teuthology.orchestra.run.smithi122.stdout:4254 punch from 0x32aa600 to 0x32b0000, (0x5a00 bytes) 2024-09-19T21:36:08.268 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:08.268 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:08.268 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:08.268 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:08.268 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:08.281 INFO:teuthology.orchestra.run.smithi122.stdout:4257 write 0xd139e00 thru 0xd148b23 (0xed24 bytes) 2024-09-19T21:36:08.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:08.289 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:08.284+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:08.340 INFO:teuthology.orchestra.run.smithi122.stdout:4258 write 0xe795600 thru 0xe79f505 (0x9f06 bytes) 2024-09-19T21:36:08.342 INFO:teuthology.orchestra.run.smithi122.stdout:4261 read 0x7837a00 thru 0x7841090 (0x9691 bytes) 2024-09-19T21:36:08.344 INFO:teuthology.orchestra.run.smithi122.stdout:4267 read 0x72c9e00 thru 0x72d5957 (0xbb58 bytes) 2024-09-19T21:36:08.388 INFO:teuthology.orchestra.run.smithi122.stdout:4268 trunc from 0xe79f600 to 0x5e04200 2024-09-19T21:36:08.390 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:08.384+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:08.408 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.677491939000006, 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:08.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:08.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:08.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.075536695, 2024-09-19T21:36:08.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:08.421 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.075124347999999, 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:36:08.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:36:08.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.075107446000001, 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:36:08.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.075090713000002, 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.075009660999999, 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:36:08.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:36:08.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:36:08.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074992931000001, 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.440 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:36:08.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:08.444 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074976534000001, 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:08.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:08.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074960100999999, 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:08.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:08.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:08.453 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074944545000001, 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.457 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074830475999999, 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:08.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074813512999999, 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:08.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074797529000001, 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074780354000001, 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:08.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.476 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074763155999999, 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.480 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074746325, 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074729804, 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:08.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074707199999999, 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:08.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:08.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074690812, 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:08.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.498 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.074581931000001, 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:08.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:08.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:08.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:08.502 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:08.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:08.504 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:08.496+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:08.542 INFO:teuthology.orchestra.run.smithi122.stdout:4274 read 0x2d78c00 thru 0x2d846a5 (0xbaa6 bytes) 2024-09-19T21:36:08.545 INFO:teuthology.orchestra.run.smithi122.stdout:4275 read 0x9c6400 thru 0x9d4885 (0xe486 bytes) 2024-09-19T21:36:08.572 INFO:teuthology.orchestra.run.smithi122.stdout:4277 write 0x7a1f800 thru 0x7a2d176 (0xd977 bytes) 2024-09-19T21:36:08.574 INFO:teuthology.orchestra.run.smithi122.stdout:4279 punch from 0x4e5aa00 to 0x4e66c00, (0xc200 bytes) 2024-09-19T21:36:08.575 INFO:teuthology.orchestra.run.smithi122.stdout:4280 read 0x320cc00 thru 0x321ad8e (0xe18f bytes) 2024-09-19T21:36:08.577 INFO:teuthology.orchestra.run.smithi122.stdout:4282 punch from 0x2304400 to 0x2306800, (0x2400 bytes) 2024-09-19T21:36:08.577 INFO:teuthology.orchestra.run.smithi122.stdout:4284 read 0x73dd000 thru 0x73dd2c3 (0x2c4 bytes) 2024-09-19T21:36:08.595 INFO:teuthology.orchestra.run.smithi122.stdout:4286 trunc from 0x7a2d200 to 0xe871000 2024-09-19T21:36:08.604 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:08.596+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:08.618 INFO:teuthology.orchestra.run.smithi122.stdout:4288 punch from 0x52a7600 to 0x52b5e00, (0xe800 bytes) 2024-09-19T21:36:08.620 INFO:teuthology.orchestra.run.smithi122.stdout:4290 write 0x755a800 thru 0x7567c1e (0xd41f bytes) 2024-09-19T21:36:08.620 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:08.620 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:08.620 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:08.620 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:08.621 INFO:teuthology.orchestra.run.smithi122.stdout:4294 punch from 0xb24c800 to 0xb258400, (0xbc00 bytes) 2024-09-19T21:36:08.622 INFO:teuthology.orchestra.run.smithi122.stdout:4295 write 0x4659c00 thru 0x4662d77 (0x9178 bytes) 2024-09-19T21:36:08.623 INFO:teuthology.orchestra.run.smithi122.stdout:4298 read 0xb7a1a00 thru 0xb7b07a6 (0xeda7 bytes) 2024-09-19T21:36:08.626 INFO:teuthology.orchestra.run.smithi122.stdout:4299 punch from 0x7c27800 to 0x7c36800, (0xf000 bytes) 2024-09-19T21:36:08.626 INFO:teuthology.orchestra.run.smithi122.stdout:4305 write 0x4ce200 thru 0x4cfc26 (0x1a27 bytes) 2024-09-19T21:36:08.628 INFO:teuthology.orchestra.run.smithi122.stdout:4309 read 0x960e200 thru 0x961cd13 (0xeb14 bytes) 2024-09-19T21:36:08.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:08.684 INFO:teuthology.orchestra.run.smithi122.stdout:4310 trunc from 0xe871000 to 0x8da400 2024-09-19T21:36:08.704 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:08.696+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:08.732 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:08.732 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:08.732 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:08.732 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:08.732 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:08.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:08.749 INFO:teuthology.orchestra.run.smithi122.stdout:4314 read 0x539600 thru 0x5485e1 (0xefe2 bytes) 2024-09-19T21:36:08.753 INFO:teuthology.orchestra.run.smithi122.stdout:4315 read 0x1c3c00 thru 0x1ce96f (0xad70 bytes) 2024-09-19T21:36:08.754 INFO:teuthology.orchestra.run.smithi122.stdout:4316 read 0x2c7400 thru 0x2d4189 (0xcd8a bytes) 2024-09-19T21:36:08.756 INFO:teuthology.orchestra.run.smithi122.stdout:truncating to largest ever: 0xee35800 2024-09-19T21:36:08.784 INFO:teuthology.orchestra.run.smithi122.stdout:4319 trunc from 0x8da400 to 0xee35800 2024-09-19T21:36:08.805 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:08.800+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:08.806 INFO:teuthology.orchestra.run.smithi122.stdout:4320 read 0xd07b800 thru 0xd07b913 (0x114 bytes) 2024-09-19T21:36:08.808 INFO:teuthology.orchestra.run.smithi122.stdout:4323 read 0x7f00200 thru 0x7f08bc4 (0x89c5 bytes) 2024-09-19T21:36:08.809 INFO:teuthology.orchestra.run.smithi122.stdout:4325 read 0x205b400 thru 0x205d094 (0x1c95 bytes) 2024-09-19T21:36:08.809 INFO:teuthology.orchestra.run.smithi122.stdout:4327 write 0xc33d400 thru 0xc33dbad (0x7ae bytes) 2024-09-19T21:36:08.810 INFO:teuthology.orchestra.run.smithi122.stdout:4329 write 0xcd92800 thru 0xcd9d675 (0xae76 bytes) 2024-09-19T21:36:08.811 INFO:teuthology.orchestra.run.smithi122.stdout:4330 write 0x6419a00 thru 0x6422394 (0x8995 bytes) 2024-09-19T21:36:08.812 INFO:teuthology.orchestra.run.smithi122.stdout:4331 read 0x441f200 thru 0x442d048 (0xde49 bytes) 2024-09-19T21:36:08.813 INFO:teuthology.orchestra.run.smithi122.stdout:4332 write 0xdbc7a00 thru 0xdbd5419 (0xda1a bytes) 2024-09-19T21:36:08.815 INFO:teuthology.orchestra.run.smithi122.stdout:4333 write 0xc48a200 thru 0xc497051 (0xce52 bytes) 2024-09-19T21:36:08.825 INFO:teuthology.orchestra.run.smithi122.stdout:4337 trunc from 0xee35800 to 0xa31c200 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597853727999997, 2024-09-19T21:36:08.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597840189000003, 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597814448999998, 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:08.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597800235000001, 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:08.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:08.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:08.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597779318000001, 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:08.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:08.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:08.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597763942, 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:08.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:08.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597749014000001, 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.889 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597696394000003, 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597646816000001, 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597629601000001, 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:08.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:08.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:08.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:08.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597610463000002, 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:08.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:08.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597595615000003, 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:08.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.597572622000001, 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.596840329999999, 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.59679002, 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:08.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:08.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:08.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:08.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:08.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.596727708000003, 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:08.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.595397075000001, 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.949 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.953 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.595261497000003, 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:08.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.595184738, 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:08.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:08.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.595126014000002, 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:08.963 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:08.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:08.967 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:08.968 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:36:08.970 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:08.963+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4338 trunc from 0xa31c200 to 0x4a42600 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4339 punch from 0x44f4e00 to 0x4500200, (0xb400 bytes) 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4340 read 0xaa8200 thru 0xaaf1d5 (0x6fd6 bytes) 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4341 read 0x3aa1e00 thru 0x3aa8f17 (0x7118 bytes) 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4342 punch from 0x137be00 to 0x138a200, (0xe400 bytes) 2024-09-19T21:36:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:4346 write 0xa5bee00 thru 0xa5c3594 (0x4795 bytes) 2024-09-19T21:36:08.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:08.986 INFO:teuthology.orchestra.run.smithi122.stdout:4347 write 0xaa1be00 thru 0xaa26c6d (0xae6e bytes) 2024-09-19T21:36:08.991 INFO:teuthology.orchestra.run.smithi122.stdout:4348 write 0x2888c00 thru 0x289169d (0x8a9e bytes) 2024-09-19T21:36:08.992 INFO:teuthology.orchestra.run.smithi122.stdout:4349 write 0x4e58e00 thru 0x4e5ec29 (0x5e2a bytes) 2024-09-19T21:36:09.021 INFO:teuthology.orchestra.run.smithi122.stdout:4351 write 0xbeb5400 thru 0xbeb7fd6 (0x2bd7 bytes) 2024-09-19T21:36:09.025 INFO:teuthology.orchestra.run.smithi122.stdout:4352 write 0x7ee3e00 thru 0x7ee6e3a (0x303b bytes) 2024-09-19T21:36:09.026 INFO:teuthology.orchestra.run.smithi122.stdout:4353 read 0x80b7800 thru 0x80bbf89 (0x478a bytes) 2024-09-19T21:36:09.027 INFO:teuthology.orchestra.run.smithi122.stdout:4354 read 0x28d3000 thru 0x28d3b1d (0xb1e bytes) 2024-09-19T21:36:09.028 INFO:teuthology.orchestra.run.smithi122.stdout:4356 read 0xb3bf600 thru 0xb3c04fb (0xefc bytes) 2024-09-19T21:36:09.029 INFO:teuthology.orchestra.run.smithi122.stdout:4359 write 0x1b4800 thru 0x1bcc69 (0x846a bytes) 2024-09-19T21:36:09.029 INFO:teuthology.orchestra.run.smithi122.stdout:4361 read 0x5d08c00 thru 0x5d0b7ab (0x2bac bytes) 2024-09-19T21:36:09.032 INFO:teuthology.orchestra.run.smithi122.stdout:4365 read 0xad9b200 thru 0xada36c4 (0x84c5 bytes) 2024-09-19T21:36:09.032 INFO:teuthology.orchestra.run.smithi122.stdout:4366 punch from 0x188a400 to 0x188f000, (0x4c00 bytes) 2024-09-19T21:36:09.032 INFO:teuthology.orchestra.run.smithi122.stdout:4367 read 0x731000 thru 0x73ed5a (0xdd5b bytes) 2024-09-19T21:36:09.034 INFO:teuthology.orchestra.run.smithi122.stdout:4368 read 0x6785a00 thru 0x6794bb9 (0xf1ba bytes) 2024-09-19T21:36:09.034 INFO:teuthology.orchestra.run.smithi122.stdout:4369 punch from 0x9682c00 to 0x968d000, (0xa400 bytes) 2024-09-19T21:36:09.055 INFO:teuthology.orchestra.run.smithi122.stdout:4370 write 0xd001600 thru 0xd00cca9 (0xb6aa bytes) 2024-09-19T21:36:09.070 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:09.064+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:09.077 INFO:teuthology.orchestra.run.smithi122.stdout:4372 trunc from 0xd00ce00 to 0x1f28800 2024-09-19T21:36:09.126 INFO:teuthology.orchestra.run.smithi122.stdout:4373 write 0x12bc600 thru 0x12be7eb (0x21ec bytes) 2024-09-19T21:36:09.128 INFO:teuthology.orchestra.run.smithi122.stdout:4375 read 0xc85c00 thru 0xc94c7e (0xf07f bytes) 2024-09-19T21:36:09.131 INFO:teuthology.orchestra.run.smithi122.stdout:4376 punch from 0xbe800 to 0xc2400, (0x3c00 bytes) 2024-09-19T21:36:09.137 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.137 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:09.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2811 3.1 3:83e3850b:::rbd_data.11069cfc5e49.0000000000000000:head [read 512~512 out=512b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.134894+0000", 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00059945599999999995, 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00064272500000000002, 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2811 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.138 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134894+0000", 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134894+0000", 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134893+0000", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134898+0000", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4850000000000003e-06 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134906+0000", 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2230000000000007e-06 2024-09-19T21:36:09.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134912+0000", 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2480000000000001e-06 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135167+0000", 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00025461 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135384+0000", 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021704299999999999 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2812 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.134918+0000", 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00057543700000000004, 2024-09-19T21:36:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072488700000000001, 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2812 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134918+0000", 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134918+0000", 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134917+0000", 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134921+0000", 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2049999999999998e-06 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134927+0000", 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6460000000000004e-06 2024-09-19T21:36:09.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134936+0000", 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8449999999999992e-06 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135675+0000", 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00073906299999999998 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2813 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.134958+0000", 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00053552299999999999, 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00080410400000000004, 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2813 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134958+0000", 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134958+0000", 2024-09-19T21:36:09.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134957+0000", 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134961+0000", 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9169999999999999e-06 2024-09-19T21:36:09.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134970+0000", 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6219999999999998e-06 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134976+0000", 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0090000000000004e-06 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2814 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.134981+0000", 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.000511967, 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00085596200000000004, 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2814 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134981+0000", 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134981+0000", 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134981+0000", 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134989+0000", 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3860000000000007e-06 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.134999+0000", 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.7620000000000007e-06 2024-09-19T21:36:09.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135005+0000", 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8259999999999998e-06 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2815 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.135013+0000", 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00048087299999999999, 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00090456, 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2815 2024-09-19T21:36:09.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135013+0000", 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135013+0000", 2024-09-19T21:36:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135012+0000", 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135015+0000", 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1949999999999999e-06 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135022+0000", 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.905e-06 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135028+0000", 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1770000000000003e-06 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2816 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.135033+0000", 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.00046007600000000002, 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00095757199999999996, 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2816 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135033+0000", 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135033+0000", 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135033+0000", 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135036+0000", 2024-09-19T21:36:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.948e-06 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135042+0000", 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4030000000000004e-06 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135048+0000", 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5230000000000003e-06 2024-09-19T21:36:09.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5646.0:2817 3.1 3.d0a1c7c1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:09.135053+0000", 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.000440622, 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001012733, 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5646", 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2020312869", 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 2817 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135053+0000", 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135053+0000", 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135052+0000", 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135056+0000", 2024-09-19T21:36:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2880000000000001e-06 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135062+0000", 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4459999999999998e-06 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:09.135067+0000", 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3970000000000002e-06 2024-09-19T21:36:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 7 2024-09-19T21:36:09.168 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:4377 trunc from 0x1f28800 to 0xbd46c00 2024-09-19T21:36:09.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:09.171 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:09.167+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:09.175 INFO:teuthology.orchestra.run.smithi122.stdout:4383 punch from 0xb7e7800 to 0xb7f1200, (0x9a00 bytes) 2024-09-19T21:36:09.178 INFO:teuthology.orchestra.run.smithi122.stdout:4384 read 0x42ab800 thru 0x42b312f (0x7930 bytes) 2024-09-19T21:36:09.179 INFO:teuthology.orchestra.run.smithi122.stdout:4387 read 0x8694a00 thru 0x869eda8 (0xa3a9 bytes) 2024-09-19T21:36:09.180 INFO:teuthology.orchestra.run.smithi122.stdout:4388 write 0x38f0c00 thru 0x38f907d (0x847e bytes) 2024-09-19T21:36:09.208 INFO:teuthology.orchestra.run.smithi122.stdout:4389 write 0xe0ff800 thru 0xe10899e (0x919f bytes) 2024-09-19T21:36:09.213 INFO:teuthology.orchestra.run.smithi122.stdout:4391 write 0x66c4600 thru 0x66d4583 (0xff84 bytes) 2024-09-19T21:36:09.214 INFO:teuthology.orchestra.run.smithi122.stdout:4392 read 0x3c3b600 thru 0x3c4590b (0xa30c bytes) 2024-09-19T21:36:09.215 INFO:teuthology.orchestra.run.smithi122.stdout:4393 punch from 0x30e400 to 0x31dc00, (0xf800 bytes) 2024-09-19T21:36:09.217 INFO:teuthology.orchestra.run.smithi122.stdout:4394 punch from 0xc8e000 to 0xc94600, (0x6600 bytes) 2024-09-19T21:36:09.217 INFO:teuthology.orchestra.run.smithi122.stdout:4397 read 0x69b1200 thru 0x69b658c (0x538d bytes) 2024-09-19T21:36:09.233 INFO:teuthology.orchestra.run.smithi122.stdout:4399 trunc from 0xe108a00 to 0x6945200 2024-09-19T21:36:09.265 INFO:teuthology.orchestra.run.smithi122.stdout:4402 write 0xfb8400 thru 0xfc1513 (0x9114 bytes) 2024-09-19T21:36:09.272 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:09.267+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:09.285 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.285 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:09.285 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:09.285 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:09.285 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:09.300 INFO:teuthology.orchestra.run.smithi122.stdout:4403 write 0xc75ec00 thru 0xc76c137 (0xd538 bytes) 2024-09-19T21:36:09.302 INFO:teuthology.orchestra.run.smithi122.stdout:4404 write 0xa7f6a00 thru 0xa7f8b93 (0x2194 bytes) 2024-09-19T21:36:09.302 INFO:teuthology.orchestra.run.smithi122.stdout:4406 write 0x78dde00 thru 0x78eba77 (0xdc78 bytes) 2024-09-19T21:36:09.303 INFO:teuthology.orchestra.run.smithi122.stdout:4407 punch from 0x37ea200 to 0x37f7400, (0xd200 bytes) 2024-09-19T21:36:09.305 INFO:teuthology.orchestra.run.smithi122.stdout:4410 write 0x6f9a600 thru 0x6fa7e48 (0xd849 bytes) 2024-09-19T21:36:09.306 INFO:teuthology.orchestra.run.smithi122.stdout:4411 punch from 0x42ae000 to 0x42b4a00, (0x6a00 bytes) 2024-09-19T21:36:09.307 INFO:teuthology.orchestra.run.smithi122.stdout:4414 write 0xb516000 thru 0xb5190db (0x30dc bytes) 2024-09-19T21:36:09.308 INFO:teuthology.orchestra.run.smithi122.stdout:4415 write 0x13cba00 thru 0x13d7e0c (0xc40d bytes) 2024-09-19T21:36:09.309 INFO:teuthology.orchestra.run.smithi122.stdout:4416 punch from 0x5c9b200 to 0x5ca6600, (0xb400 bytes) 2024-09-19T21:36:09.310 INFO:teuthology.orchestra.run.smithi122.stdout:4417 write 0x79ffa00 thru 0x7a03526 (0x3b27 bytes) 2024-09-19T21:36:09.311 INFO:teuthology.orchestra.run.smithi122.stdout:4418 write 0x9321000 thru 0x9330682 (0xf683 bytes) 2024-09-19T21:36:09.312 INFO:teuthology.orchestra.run.smithi122.stdout:4419 read 0x7e66600 thru 0x7e6885a (0x225b bytes) 2024-09-19T21:36:09.314 INFO:teuthology.orchestra.run.smithi122.stdout:4420 write 0x3e95e00 thru 0x3e9b3d6 (0x55d7 bytes) 2024-09-19T21:36:09.314 INFO:teuthology.orchestra.run.smithi122.stdout:4422 write 0x6c36200 thru 0x6c42d18 (0xcb19 bytes) 2024-09-19T21:36:09.315 INFO:teuthology.orchestra.run.smithi122.stdout:4424 read 0x6fd8400 thru 0x6fd985b (0x145c bytes) 2024-09-19T21:36:09.322 INFO:teuthology.orchestra.run.smithi122.stdout:4425 trunc from 0xc76c200 to 0x9ed8800 2024-09-19T21:36:09.345 INFO:teuthology.orchestra.run.smithi122.stdout:4428 read 0x2304c00 thru 0x230e728 (0x9b29 bytes) 2024-09-19T21:36:09.349 INFO:teuthology.orchestra.run.smithi122.stdout:4429 read 0x1680e00 thru 0x168bfce (0xb1cf bytes) 2024-09-19T21:36:09.351 INFO:teuthology.orchestra.run.smithi122.stdout:4430 read 0x6882000 thru 0x6882cd1 (0xcd2 bytes) 2024-09-19T21:36:09.370 INFO:teuthology.orchestra.run.smithi122.stdout:4432 trunc from 0x9ed8800 to 0x13b4200 2024-09-19T21:36:09.373 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:09.367+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.205011114999998, 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:09.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.204431204000002, 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:09.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.204368557000002, 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:09.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.204302237, 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203598559, 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.426 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:09.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203525737, 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:09.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:09.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203511999, 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203494310000004, 2024-09-19T21:36:09.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:09.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203443923999998, 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203417035000001, 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203376458999998, 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203354845, 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:09.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.462 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 41.203335645999999, 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:09.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:09.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.714618152, 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.714350875000001, 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:09.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.714018043999999, 2024-09-19T21:36:09.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.713470900000001, 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:09.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:09.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:09.486 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.713394562000001, 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:09.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:09.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.713062286, 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:09.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.71275936, 2024-09-19T21:36:09.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:09.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:09.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.501 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.502 INFO:teuthology.orchestra.run.smithi122.stdout:4434 write 0x33d5e00 thru 0x33dece1 (0x8ee2 bytes) 2024-09-19T21:36:09.502 INFO:teuthology.orchestra.run.smithi122.stdout:4435 write 0x8b38200 thru 0x8b45676 (0xd477 bytes) 2024-09-19T21:36:09.502 INFO:teuthology.orchestra.run.smithi122.stdout:4437 read 0x1939000 thru 0x19415dd (0x85de bytes) 2024-09-19T21:36:09.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:09.503 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:09.496+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:09.519 INFO:teuthology.orchestra.run.smithi122.stdout:4440 write 0xbf71a00 thru 0xbf77d7e (0x637f bytes) 2024-09-19T21:36:09.522 INFO:teuthology.orchestra.run.smithi122.stdout:4443 read 0x740c000 thru 0x740d8fc (0x18fd bytes) 2024-09-19T21:36:09.522 INFO:teuthology.orchestra.run.smithi122.stdout:4444 punch from 0xb714a00 to 0xb71b600, (0x6c00 bytes) 2024-09-19T21:36:09.523 INFO:teuthology.orchestra.run.smithi122.stdout:4445 read 0x7cd5800 thru 0x7cd84ce (0x2ccf bytes) 2024-09-19T21:36:09.526 INFO:teuthology.orchestra.run.smithi122.stdout:4449 read 0x8bd6800 thru 0x8be4031 (0xd832 bytes) 2024-09-19T21:36:09.533 INFO:teuthology.orchestra.run.smithi122.stdout:4455 trunc from 0xbf77e00 to 0xe216a00 2024-09-19T21:36:09.603 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:09.596+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:09.614 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:09.614 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:09.704 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:09.699+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:09.731 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.731 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:09.731 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:09.731 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:09.731 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:09.783 INFO:teuthology.orchestra.run.smithi122.stdout:4458 trunc from 0xe216a00 to 0x3534200 2024-09-19T21:36:09.805 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:09.800+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:09.841 INFO:teuthology.orchestra.run.smithi122.stdout:4461 punch from 0x5e5000 to 0x5e6a00, (0x1a00 bytes) 2024-09-19T21:36:09.855 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.101644555999997, 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.856 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:09.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:09.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.47430516, 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:09.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474252694, 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:09.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474222146999999, 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:09.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:09.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474199550000002, 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:09.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474177097999998, 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474162801999999, 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:09.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:09.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:09.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474146175000001, 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.892 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:09.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474127539000001, 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:09.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474100874000001, 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:09.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474090662999998, 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.907 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:09.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.474078176999999, 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:09.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:09.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470203587, 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470186068, 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:09.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470148514000002, 2024-09-19T21:36:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:09.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:09.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470130671, 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470118222, 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.47010753, 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:09.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.470095953000001, 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:09.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:09.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 29.054123997000001, 2024-09-19T21:36:09.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:09.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:09.956 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:09.959 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:36:09.959 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:09.952+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:09.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4462 write 0xe3aba00 thru 0xe3b0d72 (0x5373 bytes) 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4465 write 0xe26c00 thru 0xe31675 (0xaa76 bytes) 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4469 read 0x9bc1800 thru 0x9bccc9b (0xb49c bytes) 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4471 read 0x1c84400 thru 0x1c889f3 (0x45f4 bytes) 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4473 write 0xab5ba00 thru 0xab67a4e (0xc04f bytes) 2024-09-19T21:36:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:4474 punch from 0xa5b9200 to 0xa5c1600, (0x8400 bytes) 2024-09-19T21:36:09.961 INFO:teuthology.orchestra.run.smithi122.stdout:4478 read 0x3147800 thru 0x3153a50 (0xc251 bytes) 2024-09-19T21:36:09.961 INFO:teuthology.orchestra.run.smithi122.stdout:4479 write 0x5f6fa00 thru 0x5f7646d (0x6a6e bytes) 2024-09-19T21:36:09.961 INFO:teuthology.orchestra.run.smithi122.stdout:4481 punch from 0xc3bd800 to 0xc3c9600, (0xbe00 bytes) 2024-09-19T21:36:09.961 INFO:teuthology.orchestra.run.smithi122.stdout:4483 trunc from 0xe3b0e00 to 0x5474800 2024-09-19T21:36:09.973 INFO:teuthology.orchestra.run.smithi122.stdout:4484 read 0x378600 thru 0x3843d8 (0xbdd9 bytes) 2024-09-19T21:36:09.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:10.017 INFO:teuthology.orchestra.run.smithi122.stdout:4487 write 0xe735800 thru 0xe74372b (0xdf2c bytes) 2024-09-19T21:36:10.043 INFO:teuthology.orchestra.run.smithi122.stdout:4489 trunc from 0xe743800 to 0x3db2400 2024-09-19T21:36:10.060 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:10.055+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:10.093 INFO:teuthology.orchestra.run.smithi122.stdout:4491 read 0x1ac4600 thru 0x1acfe95 (0xb896 bytes) 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.5646.0:4011 3.e e85/23 3:74a21184:::rbd_header.11069cfc5e49:head v 85'326, mlcod=85'324)", 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:10.123132+0000", 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00069176899999999996, 2024-09-19T21:36:10.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00072727499999999999, 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123132+0000", 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123132+0000", 2024-09-19T21:36:10.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123131+0000", 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123139+0000", 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2460000000000003e-06 2024-09-19T21:36:10.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123154+0000", 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4395e-05 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123161+0000", 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9269999999999999e-06 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123220+0000", 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9483e-05 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:10.123252+0000", 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1958999999999997e-05 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:10.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:10.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:10.129 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:10.129 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:36:10.129 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:10.131 INFO:teuthology.orchestra.run.smithi122.stdout:4494 write 0xd55f200 thru 0xd56ea1d (0xf81e bytes) 2024-09-19T21:36:10.133 INFO:teuthology.orchestra.run.smithi122.stdout:4495 punch from 0x50d7e00 to 0x50e5a00, (0xdc00 bytes) 2024-09-19T21:36:10.134 INFO:teuthology.orchestra.run.smithi122.stdout:4496 read 0x252aa00 thru 0x252de4a (0x344b bytes) 2024-09-19T21:36:10.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:10.139 INFO:teuthology.orchestra.run.smithi122.stdout:4499 trunc from 0xd56ec00 to 0xdd92200 2024-09-19T21:36:10.143 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:10.155 INFO:teuthology.orchestra.run.smithi122.stdout:4500 read 0x3175600 thru 0x317bd55 (0x6756 bytes) 2024-09-19T21:36:10.158 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:10.155+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:10.166 INFO:teuthology.orchestra.run.smithi122.stdout:4502 trunc from 0xdd92200 to 0x9cf3600 2024-09-19T21:36:10.166 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:10.190 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:10.193 INFO:teuthology.orchestra.run.smithi122.stdout:4503 write 0x3cdda00 thru 0x3ce9325 (0xb926 bytes) 2024-09-19T21:36:10.196 INFO:teuthology.orchestra.run.smithi122.stdout:4504 write 0x9563200 thru 0x9572146 (0xef47 bytes) 2024-09-19T21:36:10.197 INFO:teuthology.orchestra.run.smithi122.stdout:4506 write 0x3cfb000 thru 0x3d07bf8 (0xcbf9 bytes) 2024-09-19T21:36:10.197 INFO:teuthology.orchestra.run.smithi122.stdout:4507 read 0x3ed5a00 thru 0x3ee3673 (0xdc74 bytes) 2024-09-19T21:36:10.212 INFO:teuthology.orchestra.run.smithi122.stdout:4509 trunc from 0x9cf3600 to 0x3d4c200 2024-09-19T21:36:10.227 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:10.254 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.254 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.258 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:10.260 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:10.252+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:10.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:10.281 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:10.283 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:10.284 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:10.286 INFO:teuthology.orchestra.run.smithi122.stdout:4511 write 0xc5fb400 thru 0xc604ddb (0x99dc bytes) 2024-09-19T21:36:10.288 INFO:teuthology.orchestra.run.smithi122.stdout:4512 write 0x9ba000 thru 0x9c8e1e (0xee1f bytes) 2024-09-19T21:36:10.289 INFO:teuthology.orchestra.run.smithi122.stdout:4516 read 0x2e7a600 thru 0x2e7cd9f (0x27a0 bytes) 2024-09-19T21:36:10.290 INFO:teuthology.orchestra.run.smithi122.stdout:4517 write 0xa675c00 thru 0xa682593 (0xc994 bytes) 2024-09-19T21:36:10.297 INFO:teuthology.orchestra.run.smithi122.stdout:4520 trunc from 0xc604e00 to 0xec9d000 2024-09-19T21:36:10.319 INFO:teuthology.orchestra.run.smithi122.stdout:4522 read 0x78ba800 thru 0x78c06c1 (0x5ec2 bytes) 2024-09-19T21:36:10.321 INFO:teuthology.orchestra.run.smithi122.stdout:4524 read 0xd5c9000 thru 0xd5d65f3 (0xd5f4 bytes) 2024-09-19T21:36:10.324 INFO:teuthology.orchestra.run.smithi122.stdout:4526 punch from 0xc397200 to 0xc3a5400, (0xe200 bytes) 2024-09-19T21:36:10.325 INFO:teuthology.orchestra.run.smithi122.stdout:4527 read 0xa333c00 thru 0xa33956b (0x596c bytes) 2024-09-19T21:36:10.328 INFO:teuthology.orchestra.run.smithi122.stdout:4528 write 0x652a000 thru 0x65390f1 (0xf0f2 bytes) 2024-09-19T21:36:10.329 INFO:teuthology.orchestra.run.smithi122.stdout:4529 punch from 0x353e800 to 0x354d200, (0xea00 bytes) 2024-09-19T21:36:10.330 INFO:teuthology.orchestra.run.smithi122.stdout:4530 write 0x54f5800 thru 0x54f5a14 (0x215 bytes) 2024-09-19T21:36:10.333 INFO:teuthology.orchestra.run.smithi122.stdout:4531 punch from 0xb2db800 to 0xb2e7400, (0xbc00 bytes) 2024-09-19T21:36:10.333 INFO:teuthology.orchestra.run.smithi122.stdout:4532 write 0x9c9ca00 thru 0x9cac2e1 (0xf8e2 bytes) 2024-09-19T21:36:10.334 INFO:teuthology.orchestra.run.smithi122.stdout:4533 write 0x6826200 thru 0x6833aeb (0xd8ec bytes) 2024-09-19T21:36:10.336 INFO:teuthology.orchestra.run.smithi122.stdout:4534 read 0xc758400 thru 0xc75bcdf (0x38e0 bytes) 2024-09-19T21:36:10.337 INFO:teuthology.orchestra.run.smithi122.stdout:4535 write 0x7ca7a00 thru 0x7cb4046 (0xc647 bytes) 2024-09-19T21:36:10.338 INFO:teuthology.orchestra.run.smithi122.stdout:4536 read 0x2e3fa00 thru 0x2e4ef7e (0xf57f bytes) 2024-09-19T21:36:10.343 INFO:teuthology.orchestra.run.smithi122.stdout:4538 read 0x5cfe400 thru 0x5d09746 (0xb347 bytes) 2024-09-19T21:36:10.361 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:10.356+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:10.367 INFO:teuthology.orchestra.run.smithi122.stdout:4540 trunc from 0xec9d000 to 0x3e37a00 2024-09-19T21:36:10.400 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.400 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:10.422 INFO:teuthology.orchestra.run.smithi122.stdout:4541 write 0x42c000 thru 0x4366fa (0xa6fb bytes) 2024-09-19T21:36:10.462 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:10.456+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:10.468 INFO:teuthology.orchestra.run.smithi122.stdout:4542 write 0xdafa400 thru 0xdafb276 (0xe77 bytes) 2024-09-19T21:36:10.471 INFO:teuthology.orchestra.run.smithi122.stdout:4546 read 0xa760400 thru 0xa763892 (0x3493 bytes) 2024-09-19T21:36:10.471 INFO:teuthology.orchestra.run.smithi122.stdout:4548 read 0x811be00 thru 0x811fe44 (0x4045 bytes) 2024-09-19T21:36:10.472 INFO:teuthology.orchestra.run.smithi122.stdout:4549 read 0x4850e00 thru 0x4858c96 (0x7e97 bytes) 2024-09-19T21:36:10.473 INFO:teuthology.orchestra.run.smithi122.stdout:4552 write 0x66d9800 thru 0x66da55a (0xd5b bytes) 2024-09-19T21:36:10.474 INFO:teuthology.orchestra.run.smithi122.stdout:4553 punch from 0xac03200 to 0xac04800, (0x1600 bytes) 2024-09-19T21:36:10.474 INFO:teuthology.orchestra.run.smithi122.stdout:4554 write 0xda80c00 thru 0xda834be (0x28bf bytes) 2024-09-19T21:36:10.475 INFO:teuthology.orchestra.run.smithi122.stdout:4555 read 0x2d06e00 thru 0x2d0c49c (0x569d bytes) 2024-09-19T21:36:10.482 INFO:teuthology.orchestra.run.smithi122.stdout:4556 trunc from 0xdafb400 to 0xc6a6400 2024-09-19T21:36:10.503 INFO:teuthology.orchestra.run.smithi122.stdout:4558 flatten 2024-09-19T21:36:10.528 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.529 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.529 INFO:teuthology.orchestra.run.smithi122.stdout:4559 punch from 0x990ee00 to 0x990f400, (0x600 bytes) 2024-09-19T21:36:10.530 INFO:teuthology.orchestra.run.smithi122.stdout:4561 read 0x172d400 thru 0x173109f (0x3ca0 bytes) 2024-09-19T21:36:10.531 INFO:teuthology.orchestra.run.smithi122.stdout:4562 punch from 0x6f6f400 to 0x6f72200, (0x2e00 bytes) 2024-09-19T21:36:10.531 INFO:teuthology.orchestra.run.smithi122.stdout:4564 read 0x62d8000 thru 0x62e32e6 (0xb2e7 bytes) 2024-09-19T21:36:10.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:10.541 INFO:teuthology.orchestra.run.smithi122.stdout:4566 write 0x3cafc00 thru 0x3cbae9d (0xb29e bytes) 2024-09-19T21:36:10.542 INFO:teuthology.orchestra.run.smithi122.stdout:4567 write 0x5526400 thru 0x55313c0 (0xafc1 bytes) 2024-09-19T21:36:10.544 INFO:teuthology.orchestra.run.smithi122.stdout:4568 read 0x9af9c00 thru 0x9affe76 (0x6277 bytes) 2024-09-19T21:36:10.544 INFO:teuthology.orchestra.run.smithi122.stdout:4571 read 0x4d79200 thru 0x4d792c2 (0xc3 bytes) 2024-09-19T21:36:10.545 INFO:teuthology.orchestra.run.smithi122.stdout:4574 read 0x25c4e00 thru 0x25cd31c (0x851d bytes) 2024-09-19T21:36:10.546 INFO:teuthology.orchestra.run.smithi122.stdout:4577 read 0xb037a00 thru 0xb0414d3 (0x9ad4 bytes) 2024-09-19T21:36:10.563 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:10.556+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:10.565 INFO:teuthology.orchestra.run.smithi122.stdout:4581 trunc from 0xc6a6400 to 0x42ed800 2024-09-19T21:36:10.623 INFO:teuthology.orchestra.run.smithi122.stdout:4582 write 0x4f4200 thru 0x502db7 (0xebb8 bytes) 2024-09-19T21:36:10.634 INFO:teuthology.orchestra.run.smithi122.stdout:4583 trunc from 0x42ed800 to 0x1c81e00 2024-09-19T21:36:10.650 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.650 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:10.664 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:10.660+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:10.701 INFO:teuthology.orchestra.run.smithi122.stdout:4584 write 0xc27dc00 thru 0xc288ea2 (0xb2a3 bytes) 2024-09-19T21:36:10.709 INFO:teuthology.orchestra.run.smithi122.stdout:4585 trunc from 0xc289000 to 0xe9a8000 2024-09-19T21:36:10.731 INFO:teuthology.orchestra.run.smithi122.stdout:4593 write 0xb15da00 thru 0xb16b80c (0xde0d bytes) 2024-09-19T21:36:10.733 INFO:teuthology.orchestra.run.smithi122.stdout:4598 punch from 0x8bb4c00 to 0x8bbd000, (0x8400 bytes) 2024-09-19T21:36:10.734 INFO:teuthology.orchestra.run.smithi122.stdout:4603 read 0xc83b000 thru 0xc83b24c (0x24d bytes) 2024-09-19T21:36:10.735 INFO:teuthology.orchestra.run.smithi122.stdout:4604 punch from 0x9df5600 to 0x9e00e00, (0xb800 bytes) 2024-09-19T21:36:10.735 INFO:teuthology.orchestra.run.smithi122.stdout:4605 write 0xd77ce00 thru 0xd78c4ca (0xf6cb bytes) 2024-09-19T21:36:10.740 INFO:teuthology.orchestra.run.smithi122.stdout:4606 punch from 0x95d0800 to 0x95d7800, (0x7000 bytes) 2024-09-19T21:36:10.741 INFO:teuthology.orchestra.run.smithi122.stdout:4609 write 0x901800 thru 0x90ca4f (0xb250 bytes) 2024-09-19T21:36:10.742 INFO:teuthology.orchestra.run.smithi122.stdout:4610 write 0x66aec00 thru 0x66b4890 (0x5c91 bytes) 2024-09-19T21:36:10.765 INFO:teuthology.orchestra.run.smithi122.stdout:4611 write 0xeb33800 thru 0xeb3aeda (0x76db bytes) 2024-09-19T21:36:10.766 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:10.759+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:10.768 INFO:teuthology.orchestra.run.smithi122.stdout:4612 write 0xc749600 thru 0xc74b4ef (0x1ef0 bytes) 2024-09-19T21:36:10.769 INFO:teuthology.orchestra.run.smithi122.stdout:4613 read 0x63b5400 thru 0x63b5b09 (0x70a bytes) 2024-09-19T21:36:10.770 INFO:teuthology.orchestra.run.smithi122.stdout:4614 write 0xd090400 thru 0xd0914e7 (0x10e8 bytes) 2024-09-19T21:36:10.770 INFO:teuthology.orchestra.run.smithi122.stdout:4615 read 0xdd90000 thru 0xdd9a464 (0xa465 bytes) 2024-09-19T21:36:10.771 INFO:teuthology.orchestra.run.smithi122.stdout:4617 read 0x4966000 thru 0x496a8e1 (0x48e2 bytes) 2024-09-19T21:36:10.772 INFO:teuthology.orchestra.run.smithi122.stdout:4619 read 0x9f44a00 thru 0x9f48ba0 (0x41a1 bytes) 2024-09-19T21:36:10.773 INFO:teuthology.orchestra.run.smithi122.stdout:4620 write 0x42db600 thru 0x42df2dc (0x3cdd bytes) 2024-09-19T21:36:10.774 INFO:teuthology.orchestra.run.smithi122.stdout:4623 read 0xc4f3e00 thru 0xc4f8d86 (0x4f87 bytes) 2024-09-19T21:36:10.774 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.775 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.785 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:10.795 INFO:teuthology.orchestra.run.smithi122.stdout:4624 trunc from 0xeb3b000 to 0x3a2dc00 2024-09-19T21:36:10.846 INFO:teuthology.orchestra.run.smithi122.stdout:4626 read 0xa72000 thru 0xa78ac7 (0x6ac8 bytes) 2024-09-19T21:36:10.849 INFO:teuthology.orchestra.run.smithi122.stdout:4627 write 0x428800 thru 0x42c533 (0x3d34 bytes) 2024-09-19T21:36:10.850 INFO:teuthology.orchestra.run.smithi122.stdout:4630 read 0x3239000 thru 0x323face (0x6acf bytes) 2024-09-19T21:36:10.851 INFO:teuthology.orchestra.run.smithi122.stdout:4632 read 0x95a000 thru 0x9640aa (0xa0ab bytes) 2024-09-19T21:36:10.853 INFO:teuthology.orchestra.run.smithi122.stdout:4637 punch from 0x2789600 to 0x278be00, (0x2800 bytes) 2024-09-19T21:36:10.866 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:10.863+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:10.874 INFO:teuthology.orchestra.run.smithi122.stdout:4638 trunc from 0x3a2dc00 to 0xca1e800 2024-09-19T21:36:10.891 INFO:teuthology.orchestra.run.smithi122.stdout:4639 read 0x8ba7200 thru 0x8bac75d (0x555e bytes) 2024-09-19T21:36:10.893 INFO:teuthology.orchestra.run.smithi122.stdout:4640 read 0x4378a00 thru 0x43859b0 (0xcfb1 bytes) 2024-09-19T21:36:10.896 INFO:teuthology.orchestra.run.smithi122.stdout:4642 punch from 0x7dada00 to 0x7daea00, (0x1000 bytes) 2024-09-19T21:36:10.897 INFO:teuthology.orchestra.run.smithi122.stdout:4643 read 0x7389a00 thru 0x738d620 (0x3c21 bytes) 2024-09-19T21:36:10.900 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:10.901 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:10.902 INFO:teuthology.orchestra.run.smithi122.stdout:4647 trunc from 0xca1e800 to 0xb01a600 2024-09-19T21:36:10.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:10.934 INFO:teuthology.orchestra.run.smithi122.stdout:4648 trunc from 0xb01a600 to 0x8981c00 2024-09-19T21:36:10.967 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:10.960+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:10.967 INFO:teuthology.orchestra.run.smithi122.stdout:4651 read 0x858c200 thru 0x859bb58 (0xf959 bytes) 2024-09-19T21:36:10.969 INFO:teuthology.orchestra.run.smithi122.stdout:4652 punch from 0x3584a00 to 0x3594200, (0xf800 bytes) 2024-09-19T21:36:10.998 INFO:teuthology.orchestra.run.smithi122.stdout:4654 write 0xd451a00 thru 0xd45f43a (0xda3b bytes) 2024-09-19T21:36:11.022 INFO:teuthology.orchestra.run.smithi122.stdout:4655 trunc from 0xd45f600 to 0x1695e00 2024-09-19T21:36:11.064 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.064 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.067 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:11.063+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:11.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:11.101 INFO:teuthology.orchestra.run.smithi122.stdout:4660 write 0x23ace00 thru 0x23b99c7 (0xcbc8 bytes) 2024-09-19T21:36:11.104 INFO:teuthology.orchestra.run.smithi122.stdout:4661 read 0x1e20600 thru 0x1e2d122 (0xcb23 bytes) 2024-09-19T21:36:11.128 INFO:teuthology.orchestra.run.smithi122.stdout:4662 trunc from 0x23b9a00 to 0xc818000 2024-09-19T21:36:11.168 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:11.163+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:11.176 INFO:teuthology.orchestra.run.smithi122.stdout:4664 write 0xd785c00 thru 0xd78c2fd (0x66fe bytes) 2024-09-19T21:36:11.178 INFO:teuthology.orchestra.run.smithi122.stdout:4668 read 0x7930200 thru 0x7935c66 (0x5a67 bytes) 2024-09-19T21:36:11.179 INFO:teuthology.orchestra.run.smithi122.stdout:4670 punch from 0x50bb000 to 0x50c2a00, (0x7a00 bytes) 2024-09-19T21:36:11.189 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.189 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.199 INFO:teuthology.orchestra.run.smithi122.stdout:4674 trunc from 0xd78c400 to 0x4429a00 2024-09-19T21:36:11.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:11.250 INFO:teuthology.orchestra.run.smithi122.stdout:4675 punch from 0x3a08c00 to 0x3a0c600, (0x3a00 bytes) 2024-09-19T21:36:11.254 INFO:teuthology.orchestra.run.smithi122.stdout:4678 write 0xc69e00 thru 0xc72260 (0x8461 bytes) 2024-09-19T21:36:11.256 INFO:teuthology.orchestra.run.smithi122.stdout:4679 punch from 0x2b78600 to 0x2b80a00, (0x8400 bytes) 2024-09-19T21:36:11.256 INFO:teuthology.orchestra.run.smithi122.stdout:4680 read 0xa3ec00 thru 0xa4d0da (0xe4db bytes) 2024-09-19T21:36:11.258 INFO:teuthology.orchestra.run.smithi122.stdout:4681 write 0x1fa5a00 thru 0x1fab25b (0x585c bytes) 2024-09-19T21:36:11.269 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:11.264+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:11.281 INFO:teuthology.orchestra.run.smithi122.stdout:4682 write 0x5056000 thru 0x50601ed (0xa1ee bytes) 2024-09-19T21:36:11.316 INFO:teuthology.orchestra.run.smithi122.stdout:4684 write 0x7294800 thru 0x7296775 (0x1f76 bytes) 2024-09-19T21:36:11.323 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.324 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.334 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:11.349 INFO:teuthology.orchestra.run.smithi122.stdout:4685 write 0x9adcc00 thru 0x9aeb319 (0xe71a bytes) 2024-09-19T21:36:11.353 INFO:teuthology.orchestra.run.smithi122.stdout:4686 read 0x2403a00 thru 0x240ac35 (0x7236 bytes) 2024-09-19T21:36:11.354 INFO:teuthology.orchestra.run.smithi122.stdout:4688 read 0x39c3000 thru 0x39d1005 (0xe006 bytes) 2024-09-19T21:36:11.370 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:11.364+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:11.389 INFO:teuthology.orchestra.run.smithi122.stdout:4690 write 0xe4f8e00 thru 0xe4fe519 (0x571a bytes) 2024-09-19T21:36:11.392 INFO:teuthology.orchestra.run.smithi122.stdout:4692 punch from 0xa271a00 to 0xa272c00, (0x1200 bytes) 2024-09-19T21:36:11.392 INFO:teuthology.orchestra.run.smithi122.stdout:4693 write 0x35aac00 thru 0x35ac59a (0x199b bytes) 2024-09-19T21:36:11.395 INFO:teuthology.orchestra.run.smithi122.stdout:4694 write 0x502d600 thru 0x503bd30 (0xe731 bytes) 2024-09-19T21:36:11.396 INFO:teuthology.orchestra.run.smithi122.stdout:4697 write 0x75bca00 thru 0x75c1830 (0x4e31 bytes) 2024-09-19T21:36:11.397 INFO:teuthology.orchestra.run.smithi122.stdout:4698 write 0x606a00 thru 0x610180 (0x9781 bytes) 2024-09-19T21:36:11.398 INFO:teuthology.orchestra.run.smithi122.stdout:4700 write 0xb370800 thru 0xb37da06 (0xd207 bytes) 2024-09-19T21:36:11.399 INFO:teuthology.orchestra.run.smithi122.stdout:4704 write 0x5847600 thru 0x584c9f0 (0x53f1 bytes) 2024-09-19T21:36:11.400 INFO:teuthology.orchestra.run.smithi122.stdout:4706 read 0x87f6e00 thru 0x87ff6d3 (0x88d4 bytes) 2024-09-19T21:36:11.401 INFO:teuthology.orchestra.run.smithi122.stdout:4709 read 0x91d9c00 thru 0x91dcf45 (0x3346 bytes) 2024-09-19T21:36:11.402 INFO:teuthology.orchestra.run.smithi122.stdout:4710 write 0x58d3a00 thru 0x58da161 (0x6762 bytes) 2024-09-19T21:36:11.403 INFO:teuthology.orchestra.run.smithi122.stdout:4711 punch from 0x593a000 to 0x593e400, (0x4400 bytes) 2024-09-19T21:36:11.405 INFO:teuthology.orchestra.run.smithi122.stdout:4712 read 0xc41e400 thru 0xc42d190 (0xed91 bytes) 2024-09-19T21:36:11.409 INFO:teuthology.orchestra.run.smithi122.stdout:4713 write 0x41eb800 thru 0x41ee23d (0x2a3e bytes) 2024-09-19T21:36:11.410 INFO:teuthology.orchestra.run.smithi122.stdout:4714 punch from 0x4c10e00 to 0x4c18200, (0x7400 bytes) 2024-09-19T21:36:11.410 INFO:teuthology.orchestra.run.smithi122.stdout:4715 punch from 0x856d600 to 0x857a400, (0xce00 bytes) 2024-09-19T21:36:11.411 INFO:teuthology.orchestra.run.smithi122.stdout:4717 read 0xd46a600 thru 0xd46aad2 (0x4d3 bytes) 2024-09-19T21:36:11.438 INFO:teuthology.orchestra.run.smithi122.stdout:4718 trunc from 0xe4fe600 to 0x3b8e00 2024-09-19T21:36:11.444 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.445 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:11.471 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:11.464+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:11.531 INFO:teuthology.orchestra.run.smithi122.stdout:4719 write 0x77f9800 thru 0x77f9a2c (0x22d bytes) 2024-09-19T21:36:11.534 INFO:teuthology.orchestra.run.smithi122.stdout:4722 punch from 0x626f600 to 0x6279000, (0x9a00 bytes) 2024-09-19T21:36:11.559 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.559 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.568 INFO:teuthology.orchestra.run.smithi122.stdout:4724 write 0xc4b3c00 thru 0xc4b7b72 (0x3f73 bytes) 2024-09-19T21:36:11.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:11.570 INFO:teuthology.orchestra.run.smithi122.stdout:4727 punch from 0x5ad6e00 to 0x5ae5a00, (0xec00 bytes) 2024-09-19T21:36:11.571 INFO:teuthology.orchestra.run.smithi122.stdout:4729 read 0x15d1200 thru 0x15d1df7 (0xbf8 bytes) 2024-09-19T21:36:11.571 INFO:teuthology.orchestra.run.smithi122.stdout:4732 punch from 0xb33c800 to 0xb34b000, (0xe800 bytes) 2024-09-19T21:36:11.572 INFO:teuthology.orchestra.run.smithi122.stdout:4733 read 0x17e5c00 thru 0x17f1d13 (0xc114 bytes) 2024-09-19T21:36:11.575 INFO:teuthology.orchestra.run.smithi122.stdout:4734 punch from 0xa698800 to 0xa69c600, (0x3e00 bytes) 2024-09-19T21:36:11.576 INFO:teuthology.orchestra.run.smithi122.stdout:4736 write 0xb56ea00 thru 0xb56eae1 (0xe2 bytes) 2024-09-19T21:36:11.577 INFO:teuthology.orchestra.run.smithi122.stdout:4737 write 0x4695600 thru 0x469d18f (0x7b90 bytes) 2024-09-19T21:36:11.578 INFO:teuthology.orchestra.run.smithi122.stdout:4739 write 0x9800000 thru 0x9809e48 (0x9e49 bytes) 2024-09-19T21:36:11.583 INFO:teuthology.orchestra.run.smithi122.stdout:4740 trunc from 0xc4b7c00 to 0xde6aa00 2024-09-19T21:36:11.607 INFO:teuthology.orchestra.run.smithi122.stdout:4743 write 0x70ddc00 thru 0x70e2a7b (0x4e7c bytes) 2024-09-19T21:36:11.609 INFO:teuthology.orchestra.run.smithi122.stdout:4744 write 0x9c9f000 thru 0x9ca2a54 (0x3a55 bytes) 2024-09-19T21:36:11.610 INFO:teuthology.orchestra.run.smithi122.stdout:4746 write 0x2cc1000 thru 0x2cccde4 (0xbde5 bytes) 2024-09-19T21:36:11.611 INFO:teuthology.orchestra.run.smithi122.stdout:4747 read 0xaf36600 thru 0xaf37b35 (0x1536 bytes) 2024-09-19T21:36:11.612 INFO:teuthology.orchestra.run.smithi122.stdout:4748 read 0x1898a00 thru 0x18a602f (0xd630 bytes) 2024-09-19T21:36:11.613 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:11.608+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:11.613 INFO:teuthology.orchestra.run.smithi122.stdout:4750 read 0xd992800 thru 0xd9a0e6a (0xe66b bytes) 2024-09-19T21:36:11.615 INFO:teuthology.orchestra.run.smithi122.stdout:4752 read 0x3181c00 thru 0x318bad5 (0x9ed6 bytes) 2024-09-19T21:36:11.616 INFO:teuthology.orchestra.run.smithi122.stdout:4754 read 0xba06a00 thru 0xba1386d (0xce6e bytes) 2024-09-19T21:36:11.620 INFO:teuthology.orchestra.run.smithi122.stdout:4756 trunc from 0xde6aa00 to 0xd1be200 2024-09-19T21:36:11.649 INFO:teuthology.orchestra.run.smithi122.stdout:4757 punch from 0xce4e600 to 0xce57a00, (0x9400 bytes) 2024-09-19T21:36:11.650 INFO:teuthology.orchestra.run.smithi122.stdout:4758 punch from 0xc2aa800 to 0xc2b4400, (0x9c00 bytes) 2024-09-19T21:36:11.650 INFO:teuthology.orchestra.run.smithi122.stdout:4759 read 0xbb19e00 thru 0xbb1d0c7 (0x32c8 bytes) 2024-09-19T21:36:11.651 INFO:teuthology.orchestra.run.smithi122.stdout:4760 read 0x872c600 thru 0x872d81b (0x121c bytes) 2024-09-19T21:36:11.652 INFO:teuthology.orchestra.run.smithi122.stdout:4762 read 0x4b21600 thru 0x4b27d66 (0x6767 bytes) 2024-09-19T21:36:11.653 INFO:teuthology.orchestra.run.smithi122.stdout:4764 read 0x7252800 thru 0x725ca26 (0xa227 bytes) 2024-09-19T21:36:11.654 INFO:teuthology.orchestra.run.smithi122.stdout:4765 punch from 0x3d29600 to 0x3d2ea00, (0x5400 bytes) 2024-09-19T21:36:11.655 INFO:teuthology.orchestra.run.smithi122.stdout:4767 read 0x5623400 thru 0x5627ae5 (0x46e6 bytes) 2024-09-19T21:36:11.672 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:11.664+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:11.680 INFO:teuthology.orchestra.run.smithi122.stdout:4768 write 0xda1cc00 thru 0xda20525 (0x3926 bytes) 2024-09-19T21:36:11.684 INFO:teuthology.orchestra.run.smithi122.stdout:4769 read 0xa8d4a00 thru 0xa8de677 (0x9c78 bytes) 2024-09-19T21:36:11.684 INFO:teuthology.orchestra.run.smithi122.stdout:4771 read 0x1a10800 thru 0x1a1b27a (0xaa7b bytes) 2024-09-19T21:36:11.685 INFO:teuthology.orchestra.run.smithi122.stdout:4772 punch from 0xb474600 to 0xb481800, (0xd200 bytes) 2024-09-19T21:36:11.699 INFO:teuthology.orchestra.run.smithi122.stdout:4773 trunc from 0xda20600 to 0x84a2200 2024-09-19T21:36:11.727 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.728 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.737 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:11.768 INFO:teuthology.orchestra.run.smithi122.stdout:4775 write 0xbb2b200 thru 0xbb2b8e6 (0x6e7 bytes) 2024-09-19T21:36:11.770 INFO:teuthology.orchestra.run.smithi122.stdout:4777 read 0x92a7200 thru 0x92afc66 (0x8a67 bytes) 2024-09-19T21:36:11.771 INFO:teuthology.orchestra.run.smithi122.stdout:4778 write 0xa12c400 thru 0xa13b8d2 (0xf4d3 bytes) 2024-09-19T21:36:11.772 INFO:teuthology.orchestra.run.smithi122.stdout:4779 punch from 0x5ea9000 to 0x5eaf200, (0x6200 bytes) 2024-09-19T21:36:11.772 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:11.767+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:11.773 INFO:teuthology.orchestra.run.smithi122.stdout:4780 write 0xa913600 thru 0xa9233d9 (0xfdda bytes) 2024-09-19T21:36:11.774 INFO:teuthology.orchestra.run.smithi122.stdout:4781 write 0x7e4b600 thru 0x7e4b906 (0x307 bytes) 2024-09-19T21:36:11.775 INFO:teuthology.orchestra.run.smithi122.stdout:4782 write 0x1425600 thru 0x143313a (0xdb3b bytes) 2024-09-19T21:36:11.776 INFO:teuthology.orchestra.run.smithi122.stdout:4783 read 0x783da00 thru 0x784a2cc (0xc8cd bytes) 2024-09-19T21:36:11.791 INFO:teuthology.orchestra.run.smithi122.stdout:4799 trunc from 0xbb2ba00 to 0x6781600 2024-09-19T21:36:11.828 INFO:teuthology.orchestra.run.smithi122.stdout:4801 punch from 0x452e00 to 0x45a400, (0x7600 bytes) 2024-09-19T21:36:11.830 INFO:teuthology.orchestra.run.smithi122.stdout:4803 read 0x2c70e00 thru 0x2c71dc0 (0xfc1 bytes) 2024-09-19T21:36:11.847 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.848 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:11.866 INFO:teuthology.orchestra.run.smithi122.stdout:4804 write 0xd47e400 thru 0xd486039 (0x7c3a bytes) 2024-09-19T21:36:11.872 INFO:teuthology.orchestra.run.smithi122.stdout:4806 punch from 0x5dd9c00 to 0x5dda800, (0xc00 bytes) 2024-09-19T21:36:11.873 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:11.867+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:11.873 INFO:teuthology.orchestra.run.smithi122.stdout:4808 read 0x3962c00 thru 0x3963476 (0x877 bytes) 2024-09-19T21:36:11.877 INFO:teuthology.orchestra.run.smithi122.stdout:4810 punch from 0xbe1000 to 0xbec600, (0xb600 bytes) 2024-09-19T21:36:11.877 INFO:teuthology.orchestra.run.smithi122.stdout:4813 write 0xba4d800 thru 0xba5a959 (0xd15a bytes) 2024-09-19T21:36:11.881 INFO:teuthology.orchestra.run.smithi122.stdout:4817 write 0x8e8200 thru 0x8f7b79 (0xf97a bytes) 2024-09-19T21:36:11.881 INFO:teuthology.orchestra.run.smithi122.stdout:4819 write 0x8863400 thru 0x886ad72 (0x7973 bytes) 2024-09-19T21:36:11.888 INFO:teuthology.orchestra.run.smithi122.stdout:4822 trunc from 0xd486200 to 0xac8dc00 2024-09-19T21:36:11.922 INFO:teuthology.orchestra.run.smithi122.stdout:4823 read 0x4a0800 thru 0x4a63f5 (0x5bf6 bytes) 2024-09-19T21:36:11.938 INFO:teuthology.orchestra.run.smithi122.stdout:4826 trunc from 0xac8dc00 to 0x54d9000 2024-09-19T21:36:11.961 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:36:11.974 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:11.974 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:11.974 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:11.967+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:11.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:11.991 INFO:teuthology.orchestra.run.smithi122.stdout:4827 trunc from 0x54d9000 to 0xcfe4e00 2024-09-19T21:36:12.011 INFO:teuthology.orchestra.run.smithi122.stdout:4830 punch from 0x3f7000 to 0x3fe000, (0x7000 bytes) 2024-09-19T21:36:12.012 INFO:teuthology.orchestra.run.smithi122.stdout:4831 write 0x65a600 thru 0x65c480 (0x1e81 bytes) 2024-09-19T21:36:12.016 INFO:teuthology.orchestra.run.smithi122.stdout:4833 write 0x52de800 thru 0x52eb61f (0xce20 bytes) 2024-09-19T21:36:12.017 INFO:teuthology.orchestra.run.smithi122.stdout:4840 write 0xce38a00 thru 0xce4088c (0x7e8d bytes) 2024-09-19T21:36:12.021 INFO:teuthology.orchestra.run.smithi122.stdout:4843 punch from 0x798c000 to 0x7994600, (0x8600 bytes) 2024-09-19T21:36:12.022 INFO:teuthology.orchestra.run.smithi122.stdout:4844 punch from 0xb4e1a00 to 0xb4e6600, (0x4c00 bytes) 2024-09-19T21:36:12.041 INFO:teuthology.orchestra.run.smithi122.stdout:4845 trunc from 0xcfe4e00 to 0x4b80e00 2024-09-19T21:36:12.074 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:12.071+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:12.088 INFO:teuthology.orchestra.run.smithi122.stdout:4846 read 0x1569400 thru 0x156f644 (0x6245 bytes) 2024-09-19T21:36:12.092 INFO:teuthology.orchestra.run.smithi122.stdout:4849 read 0x1943600 thru 0x194583d (0x223e bytes) 2024-09-19T21:36:12.093 INFO:teuthology.orchestra.run.smithi122.stdout:4852 punch from 0x28ee600 to 0x28fbc00, (0xd600 bytes) 2024-09-19T21:36:12.093 INFO:teuthology.orchestra.run.smithi122.stdout:4853 read 0x2398000 thru 0x23a56d8 (0xd6d9 bytes) 2024-09-19T21:36:12.116 INFO:teuthology.orchestra.run.smithi122.stdout:4854 write 0x4c06800 thru 0x4c068fc (0xfd bytes) 2024-09-19T21:36:12.118 INFO:teuthology.orchestra.run.smithi122.stdout:4858 read 0x4869e00 thru 0x486e6f6 (0x48f7 bytes) 2024-09-19T21:36:12.145 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:12.146 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:12.152 INFO:teuthology.orchestra.run.smithi122.stdout:4864 write 0xa910400 thru 0xa917c1c (0x781d bytes) 2024-09-19T21:36:12.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:12.159 INFO:teuthology.orchestra.run.smithi122.stdout:4867 trunc from 0xa917e00 to 0xbe46600 2024-09-19T21:36:12.175 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:12.171+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:12.183 INFO:teuthology.orchestra.run.smithi122.stdout:4868 trunc from 0xbe46600 to 0xa656600 2024-09-19T21:36:12.206 INFO:teuthology.orchestra.run.smithi122.stdout:4869 write 0x2cf7600 thru 0x2cfe40e (0x6e0f bytes) 2024-09-19T21:36:12.208 INFO:teuthology.orchestra.run.smithi122.stdout:4871 write 0x82c8800 thru 0x82d2e75 (0xa676 bytes) 2024-09-19T21:36:12.209 INFO:teuthology.orchestra.run.smithi122.stdout:4872 punch from 0x4889200 to 0x488e800, (0x5600 bytes) 2024-09-19T21:36:12.210 INFO:teuthology.orchestra.run.smithi122.stdout:4875 punch from 0xa27a600 to 0xa283a00, (0x9400 bytes) 2024-09-19T21:36:12.210 INFO:teuthology.orchestra.run.smithi122.stdout:4877 read 0x5283800 thru 0x528ef09 (0xb70a bytes) 2024-09-19T21:36:12.214 INFO:teuthology.orchestra.run.smithi122.stdout:4878 punch from 0x18bb800 to 0x18c0600, (0x4e00 bytes) 2024-09-19T21:36:12.214 INFO:teuthology.orchestra.run.smithi122.stdout:4881 write 0x687a800 thru 0x6889e85 (0xf686 bytes) 2024-09-19T21:36:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:4882 trunc from 0xa656600 to 0xd96b200 2024-09-19T21:36:12.255 INFO:teuthology.orchestra.run.smithi122.stdout:4886 trunc from 0xd96b200 to 0x9019400 2024-09-19T21:36:12.268 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:12.268 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:12.276 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:12.267+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:12.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:12.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:12.309 INFO:teuthology.orchestra.run.smithi122.stdout:4888 write 0xc83bc00 thru 0xc841e07 (0x6208 bytes) 2024-09-19T21:36:12.311 INFO:teuthology.orchestra.run.smithi122.stdout:4889 write 0x5729400 thru 0x5736a70 (0xd671 bytes) 2024-09-19T21:36:12.332 INFO:teuthology.orchestra.run.smithi122.stdout:4893 trunc from 0xc842000 to 0xede600 2024-09-19T21:36:12.377 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:12.371+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:12.388 INFO:teuthology.orchestra.run.smithi122.stdout:4894 read 0x48b000 thru 0x48d8e2 (0x28e3 bytes) 2024-09-19T21:36:12.419 INFO:teuthology.orchestra.run.smithi122.stdout:4899 write 0x56a7000 thru 0x56ad908 (0x6909 bytes) 2024-09-19T21:36:12.421 INFO:teuthology.orchestra.run.smithi122.stdout:4900 clone 30 order 24 su 524288 sc 8 2024-09-19T21:36:12.433 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:12.438 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:12.438 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:12.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:12.454 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:12.472 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:12.477 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:12.471+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:12.494 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:12.517 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:12.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:12.541 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:12.560 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:12.560 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:12.560 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:12.560 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:12.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:12.615 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:12.611+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:12.678 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:12.678 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:12.678 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:12.678 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:12.678 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:12.680 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:12.675+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:12.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:12.781 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:12.775+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.434541961999997, 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:36:12.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:36:12.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.434529654000002, 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:36:12.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:36:12.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.401798415000002, 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.805 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:36:12.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.809 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.172040721999998, 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:36:12.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:36:12.814 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.172017287000003, 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.815 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:12.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:12.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.819 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.171995559000003, 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.820 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:12.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.171974827, 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:12.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:12.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:12.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.171940391, 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:12.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.833 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:12.834 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.171897594000001, 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.835 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.836 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.837 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:12.838 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.839 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166672083999998, 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.842 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.843 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166649786000001, 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.847 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.848 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.849 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166618931000002, 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.850 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.852 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:12.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.854 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166576222000003, 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:12.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166552234999997, 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:12.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166520831, 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:12.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166495664999999, 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:12.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:12.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166450587, 2024-09-19T21:36:12.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.874 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:12.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166423842999997, 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166410550999998, 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:12.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.166393632000002, 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:12.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:12.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:12.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:12.892 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:12.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:12.894 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:36:12.895 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:12.887+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:12.994 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:12.987+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:13.006 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.006 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:13.006 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:13.006 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:13.095 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:13.091+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:13.127 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.127 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:13.127 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:13.127 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:13.127 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:13.196 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:13.187+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:13.202 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:13.202 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:13.202 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4] 2024-09-19T21:36:13.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-19T21:36:13.248 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.248 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:13.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:13.248 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 117.312852165, 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:13.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:13.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:13.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993284834000001, 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:13.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:13.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993273068000001, 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:13.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993260175000003, 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993248152, 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:13.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.271 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:13.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:13.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:13.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:13.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:13.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:13.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.277 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993230508000003, 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:13.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993214711999997, 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:13.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993202861, 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993134017000003, 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:13.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:13.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993105876999998, 2024-09-19T21:36:13.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:13.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.993082747999999, 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:13.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:13.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.978306128, 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.970969812999996, 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:13.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.315 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.970948923000002, 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.316 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:13.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:13.320 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.963873382999999, 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.325 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.325 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:13.325 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.963852838999998, 2024-09-19T21:36:13.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:13.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.783498768000001, 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:13.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.336 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:13.337 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.783488341000002, 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:13.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.341 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.783477912000002, 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:13.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:13.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.783467848999997, 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:13.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.351 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.352 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:13.354 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:13.347+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:13.454 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:13.447+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:13.504 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.504 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:13.505 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:13.505 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:13.533 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:36:13.527+0000 7eff847dd700 -1 mon.a@0(leader).osd e86 definitely_dead 0 2024-09-19T21:36:13.555 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:13.551+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:13.575 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-19T21:36:13.619 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:13.620 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:13.673 INFO:teuthology.orchestra.run.smithi122.stdout:4903 write 0x1a68800 thru 0x1a70b92 (0x8393 bytes) 2024-09-19T21:36:13.674 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:13.671+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:13.720 INFO:teuthology.orchestra.run.smithi122.stdout:4904 trunc from 0x56ada00 to 0xc41a800 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 78.006144918000004, 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:13.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:13.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:13.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.404189674000001, 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:13.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403777327, 2024-09-19T21:36:13.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:36:13.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:36:13.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:36:13.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:13.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:36:13.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:13.755 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403760425000002, 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:36:13.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:36:13.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403743691999999, 2024-09-19T21:36:13.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:13.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.40366264, 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:36:13.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403645910000002, 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.769 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:36:13.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.773 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403629512999998, 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.774 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:13.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.40361308, 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:13.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:13.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403597523999998, 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:13.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.787 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403483455, 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:13.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403466492, 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.793 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:13.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403450507999999, 2024-09-19T21:36:13.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:13.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.801 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403433332999999, 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.805 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403416135000001, 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:13.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.808 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:13.810 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403399304000001, 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:13.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:13.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403382783000001, 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403360179, 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.820 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:13.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.823 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403343791000001, 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.825 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:13.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.403234909999998, 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.829 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.830 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.831 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:13.832 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:13.833 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:13.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:13.836 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:13.827+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:4905 write 0x74f5c00 thru 0x74f617e (0x57f bytes) 2024-09-19T21:36:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:4906 trunc from 0xc41a800 to 0xa86c000 2024-09-19T21:36:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:4907 read 0x101ee00 thru 0x101fa0a (0xc0b bytes) 2024-09-19T21:36:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:4909 read 0x46a3c00 thru 0x46ad463 (0x9864 bytes) 2024-09-19T21:36:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:4917 punch from 0x7ef5400 to 0x7f02600, (0xd200 bytes) 2024-09-19T21:36:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:4919 write 0x3b0ce00 thru 0x3b17ff3 (0xb1f4 bytes) 2024-09-19T21:36:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:4920 punch from 0x58e8c00 to 0x58eda00, (0x4e00 bytes) 2024-09-19T21:36:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:4921 punch from 0x5ed3800 to 0x5eda800, (0x7000 bytes) 2024-09-19T21:36:13.837 INFO:teuthology.orchestra.run.smithi122.stdout:4922 write 0x4693000 thru 0x46999af (0x69b0 bytes) 2024-09-19T21:36:13.851 INFO:teuthology.orchestra.run.smithi122.stdout:4925 write 0xbe8d400 thru 0xbe915b1 (0x41b2 bytes) 2024-09-19T21:36:13.852 INFO:teuthology.orchestra.run.smithi122.stdout:4926 write 0x22a4600 thru 0x22b1059 (0xca5a bytes) 2024-09-19T21:36:13.856 INFO:teuthology.orchestra.run.smithi122.stdout:4928 trunc from 0xbe91600 to 0xc693400 2024-09-19T21:36:13.880 INFO:teuthology.orchestra.run.smithi122.stdout:4929 write 0x8bc2000 thru 0x8bceb0b (0xcb0c bytes) 2024-09-19T21:36:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.2. 2024-09-19T21:36:13.935 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:13.931+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:13.956 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:13.957 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:13.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:14.036 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:14.027+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:14.071 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:14.072 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:14.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:14.137 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:14.131+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:14.195 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:14.195 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937447116000001, 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.196 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:14.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.201 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937433577, 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.202 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:14.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:14.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.207 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937407837000002, 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:14.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:14.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937393622999998, 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937372705999998, 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937357329999998, 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:14.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937342401999999, 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937289782000001, 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:14.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:14.240 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937240203999998, 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:14.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:14.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937222988999999, 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:14.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:14.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937203851, 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937189003, 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:14.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.937166009999999, 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:14.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:14.269 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.936433718000004, 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:14.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.936383407999998, 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:14.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:14.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:14.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.280 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.936321096, 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:14.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:14.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:14.285 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.934990462999998, 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.286 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:14.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.934854885, 2024-09-19T21:36:14.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:14.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:14.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.934778125999998, 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:14.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:14.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:14.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.934719401999999, 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.308 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.309 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:14.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:14.311 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:14.307+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:14.412 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:14.403+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5730.0:827 3.0 3.6ea1c9c0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:13.881720+0000", 2024-09-19T21:36:14.427 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.54375036600000004, 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.54379208800000001, 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5730", 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3625230400", 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 827 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881720+0000", 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881720+0000", 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881718+0000", 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999971 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881731+0000", 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3144e-05 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881769+0000", 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8183999999999999e-05 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881792+0000", 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3025000000000001e-05 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881882+0000", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9567999999999994e-05 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881890+0000", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1359999999999997e-06 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889422+0000", 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0075324529999999997 2024-09-19T21:36:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889440+0000", 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7955e-05 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:16267 3.0 3.6ea1c9c0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:13.881891+0000", 2024-09-19T21:36:14.432 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.54357960500000002, 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.54374834400000005, 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 16267 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881891+0000", 2024-09-19T21:36:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881891+0000", 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881889+0000", 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881901+0000", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2109999999999999e-05 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881933+0000", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1634000000000002e-05 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.881953+0000", 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0533000000000001e-05 2024-09-19T21:36:14.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882038+0000", 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5061000000000004e-05 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op must wait for map", 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882049+0000", 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0553e-05 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889507+0000", 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0074578190000000001 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889514+0000", 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8859999999999998e-06 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5730.0:828 3.0 3.6ea1c9c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:13.882292+0000", 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.543179046, 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.54345745300000003, 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5730", 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3625230400", 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 828 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882292+0000", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882292+0000", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882290+0000", 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882299+0000", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8720000000000008e-06 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882318+0000", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8867000000000001e-05 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882327+0000", 2024-09-19T21:36:14.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0459999999999994e-06 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882431+0000", 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000103649 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting_for_map not empty", 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.882447+0000", 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6830999999999999e-05 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889479+0000", 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0070317560000000001 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.889487+0000", 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.9980000000000003e-06 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:16268 3.0 3.6ea1c9c0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:13.995751+0000", 2024-09-19T21:36:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.42971931099999999, 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.430105556, 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 16268 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995751+0000", 2024-09-19T21:36:14.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995751+0000", 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995750+0000", 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995759+0000", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6759999999999996e-06 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995773+0000", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.452e-05 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995789+0000", 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6022999999999999e-05 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995928+0000", 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000138484 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:13.995959+0000", 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1207999999999997e-05 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-19T21:36:14.446 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 4 2024-09-19T21:36:14.447 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:14.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:14.512 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:14.503+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:14.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:14.560 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:14.560 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:14.560 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:14.560 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:14.560 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:14.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:14.613 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:14.607+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:14.707 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:14.715 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:14.707+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:14.725 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:14.725 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.523648031999997, 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:14.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.523068121000001, 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:14.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.523005474000001, 2024-09-19T21:36:14.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:14.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522939153999999, 2024-09-19T21:36:14.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522235475999999, 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:14.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522162653999999, 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522148915999999, 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:14.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:14.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.758 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522131227000003, 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:14.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522080840999998, 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:14.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522053952, 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:14.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.775 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.522013375999997, 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:14.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:14.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.780 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.521991761999999, 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.785 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.521972562999999, 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.033255068999999, 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:14.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:14.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:14.795 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.032987792, 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:14.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.800 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.032654960999999, 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.032107817000004, 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:14.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.032031478999997, 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.811 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.031699203000002, 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:14.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.031396276999999, 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:14.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:14.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.825 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.825 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:14.825 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:14.826 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:14.827 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:14.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:14.828 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:14.819+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:14.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:36:14.927 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:14.919+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:14.987 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:14.987 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:14.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:15.041 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:15.035+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.105 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:15.128 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:15.123+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:15.224 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.224 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.229 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:15.223+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:15.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:15.330 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:15.323+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:15.342 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.342 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:15.430 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:15.427+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:15.461 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.461 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:15.531 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:15.527+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:15.577 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.577 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:15.632 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:15.627+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:15.700 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.700 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.713 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:15.758 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:15.755+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:15.823 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.824 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:15.833 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:15.827+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:15.910 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:36:15.916 INFO:teuthology.orchestra.run.smithi122.stdout:4932 read 0x3298200 thru 0x329d04b (0x4e4c bytes) 2024-09-19T21:36:15.924 INFO:teuthology.orchestra.run.smithi122.stdout:4933 read 0x3c8cc00 thru 0x3c9cb4a (0xff4b bytes) 2024-09-19T21:36:15.933 INFO:teuthology.orchestra.run.smithi122.stdout:4934 read 0x7a2b400 thru 0x7a3a9e1 (0xf5e2 bytes) 2024-09-19T21:36:15.934 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:15.927+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:15.934 INFO:teuthology.orchestra.run.smithi122.stdout:4935 write 0xbb8800 thru 0xbbc53a (0x3d3b bytes) 2024-09-19T21:36:15.964 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:15.964 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:15.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:15.981 INFO:teuthology.orchestra.run.smithi122.stdout:4936 write 0xd38ca00 thru 0xd39489d (0x7e9e bytes) 2024-09-19T21:36:15.985 INFO:teuthology.orchestra.run.smithi122.stdout:4937 write 0xc522a00 thru 0xc524fa7 (0x25a8 bytes) 2024-09-19T21:36:15.991 INFO:teuthology.orchestra.run.smithi122.stdout:4940 trunc from 0xd394a00 to 0xb17c200 2024-09-19T21:36:16.022 INFO:teuthology.orchestra.run.smithi122.stdout:4942 write 0xa3e1200 thru 0xa3efc2f (0xea30 bytes) 2024-09-19T21:36:16.024 INFO:teuthology.orchestra.run.smithi122.stdout:4946 write 0x5182200 thru 0x5183635 (0x1436 bytes) 2024-09-19T21:36:16.034 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:16.027+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:16.072 INFO:teuthology.orchestra.run.smithi122.stdout:4947 write 0xc52da00 thru 0xc535d5f (0x8360 bytes) 2024-09-19T21:36:16.077 INFO:teuthology.orchestra.run.smithi122.stdout:4949 read 0x5b14600 thru 0x5b22113 (0xdb14 bytes) 2024-09-19T21:36:16.097 INFO:teuthology.orchestra.run.smithi122.stdout:4950 trunc from 0xc535e00 to 0x50d7e00 2024-09-19T21:36:16.135 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:16.127+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:16.150 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.150 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:16.191 INFO:teuthology.orchestra.run.smithi122.stdout:4953 write 0x7166a00 thru 0x71734ba (0xcabb bytes) 2024-09-19T21:36:16.234 INFO:teuthology.orchestra.run.smithi122.stdout:4954 write 0xcb2f400 thru 0xcb37008 (0x7c09 bytes) 2024-09-19T21:36:16.234 INFO:teuthology.orchestra.run.smithi122.stdout:4956 read 0x1fa4600 thru 0x1faefac (0xa9ad bytes) 2024-09-19T21:36:16.236 INFO:teuthology.orchestra.run.smithi122.stdout:4959 write 0xb5cf000 thru 0xb5d399b (0x499c bytes) 2024-09-19T21:36:16.236 INFO:teuthology.orchestra.run.smithi122.stdout:4960 read 0xae2a600 thru 0xae3265a (0x805b bytes) 2024-09-19T21:36:16.237 INFO:teuthology.orchestra.run.smithi122.stdout:4961 clone 31 order 24 su 1048576 sc 13 2024-09-19T21:36:16.237 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:16.231+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:16.285 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.286 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:16.338 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:16.331+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:16.418 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.418 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:16.477 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:16.471+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:16.539 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:16.535+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:16.559 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.559 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:16.640 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:16.631+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:16.688 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.688 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:16.741 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:16.735+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:16.802 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.802 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:16.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:16.857 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:16.851+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:16.943 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:16.939+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:16.974 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:16.974 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:16.986 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:16.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:17.044 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:17.039+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:17.096 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:17.096 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:17.107 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:17.145 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:17.139+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:17.218 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:17.240 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:17.246 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:17.239+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:17.264 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:17.269 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:17.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:17.293 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:17.317 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:17.319 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:17.374 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:17.371+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:17.381 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:17.382 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:17.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:17.449 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:17.447+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:17.500 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:17.500 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:17.500 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:17.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:17.549 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:17.543+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:17.622 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 81.869573668000001, 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242234271999997, 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242181805999998, 2024-09-19T21:36:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:17.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:17.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242151259000003, 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:17.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:17.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242128661999999, 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242106210000003, 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242091914, 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242075286999999, 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242056650999999, 2024-09-19T21:36:17.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242029985999999, 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.671 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:17.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.675 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242019775000003, 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.242007289, 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.681 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238132698999998, 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238115180000001, 2024-09-19T21:36:17.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238077625999999, 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238059782999997, 2024-09-19T21:36:17.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:17.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238047334000001, 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:17.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:17.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:17.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238036641999997, 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.710 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:17.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 37.238025065000002, 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 36.822053109000002, 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:17.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:17.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:17.724 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:17.726 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:17.723+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:17.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:17.828 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:17.823+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:17.859 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:17.859 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:17.859 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:17.860 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:17.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:17.928 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:17.919+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:17.935 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:36:17.983 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:17.983 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:17.983 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:17.984 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:17.984 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:18.028 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:18.019+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.115 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.754447306000003, 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:36:18.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:36:18.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:36:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.121 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.754434998000001, 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.122 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:36:18.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:36:18.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.721703759, 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.129 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:18.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491946065999997, 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:36:18.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:36:18.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:36:18.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:18.139 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491922631000001, 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:18.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:18.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491900903000001, 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.145 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:18.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491880170999998, 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:18.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.154 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491845734999998, 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:18.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:18.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:18.159 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.491802937999999, 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:18.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:18.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486577427999997, 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:18.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:18.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486555129999999, 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:18.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:18.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486524275000001, 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.174 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:18.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486481566000002, 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:18.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486457579000003, 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:18.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486426174999998, 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:18.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:18.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:18.192 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486401008999998, 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:18.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:18.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486355930999999, 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:18.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486329187000003, 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:18.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:18.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486315894999997, 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.486298976, 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.215 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:18.217 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:18.207+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:18.318 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:18.311+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:18.326 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.326 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:18.326 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:18.326 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:18.418 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:18.411+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:18.488 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.488 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:18.488 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:18.488 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:18.488 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:18.519 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:18.515+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:18.606 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 122.670913042, 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:18.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:18.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.614 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351345711, 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:18.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:18.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351333945, 2024-09-19T21:36:18.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:18.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351321052000003, 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:18.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351309028999999, 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:18.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:18.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:18.634 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351291385000003, 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:18.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:18.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351275588999997, 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:18.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351263738, 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.644 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:18.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351194894000002, 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:18.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:18.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:18.654 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351166753999998, 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:18.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:18.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.660 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.351143624999999, 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:18.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.665 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.336367005, 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.329030690000003, 2024-09-19T21:36:18.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:18.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.329009800000001, 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:18.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.321934259999999, 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.680 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:18.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:18.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.321913715999997, 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.141559645000001, 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.690 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:18.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.141549218000002, 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:18.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:18.699 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.141538789000002, 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:18.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:18.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.141528725999997, 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.705 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:18.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:18.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:18.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:18.710 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:18.710 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:18.712 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:18.707+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:18.812 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:18.807+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:18.835 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.835 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:18.835 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:18.835 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.840 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:36:18.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:18.906 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4] out_osds: [5, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:18.907 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:18.907 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-19T21:36:18.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-19T21:36:18.948 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:18.948 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:18.948 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:18.948 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:18.948 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:18.948 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:18.943+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:18.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:19.014 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:19.007+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:19.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-19T21:36:19.114 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:19.111+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:19.131 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone30 2024-09-19T21:36:19.155 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:19.155 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.423150571999997, 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.156 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:19.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:19.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.821195328000002, 2024-09-19T21:36:19.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:19.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:19.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:19.167 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820782981000001, 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:19.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820766079000002, 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:36:19.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820749345999999, 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:19.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:19.181 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820668294000001, 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:36:19.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820651564000002, 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:19.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:36:19.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820635166999999, 2024-09-19T21:36:19.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:19.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:19.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820618734, 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.196 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820603177999999, 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820489109, 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820472146, 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820456161999999, 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820438987000003, 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820421789000001, 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820404957999997, 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820388436999998, 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820365832999997, 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820349444999998, 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.820240564000002, 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:19.255 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:19.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:19.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-19T21:36:19.257 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:19.251+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:19.332 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:19.356 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:19.351+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:19.381 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:19.381 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:19.381 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:19.381 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:19.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:19.457 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:19.451+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:19.464 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-19T21:36:19.468 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:19.486 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:19.498 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:19.505 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:19.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:19.524 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:19.545 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:19.558 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:19.551+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:19.566 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:19.624 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:19.624 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:19.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:19.670 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:19.667+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:19.740 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:19.740 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:19.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:19.760 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:19.755+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:19.847 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-19T21:36:19.861 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:19.855+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:19.862 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:19.863 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:19.873 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:19.963 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:19.959+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:20.000 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.000 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:20.063 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:20.055+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:20.115 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.116 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:20.164 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:20.163+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:20.172 INFO:teuthology.orchestra.run.smithi122.stdout:4963 clone 32 order 19 su 131072 sc 14 2024-09-19T21:36:20.265 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:20.259+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:20.276 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.276 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:20.366 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:20.359+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:20.391 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.391 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:20.466 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:20.463+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:20.519 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.519 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:20.567 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:20.559+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:20.635 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.635 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:20.668 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:20.663+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:20.752 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.752 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:20.769 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:20.763+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:20.854 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:36:20.869 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:20.870 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:20.870 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:20.867+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:20.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:20.971 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:20.967+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:21.028 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.028 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:21.078 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone31 from 0xcb37200 (overlap 0xcb37200) to 0x5149400 2024-09-19T21:36:21.082 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:21.079+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:21.161 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.161 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:21.173 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:21.167+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:21.272 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.273 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.274 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:21.267+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:21.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:21.375 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:21.371+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:21.392 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.392 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:21.477 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:21.471+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:21.510 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.510 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:21.577 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:21.571+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:21.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:21.642 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.642 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:21.678 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:21.671+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:21.730 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:21.753 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:21.761 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:21.761 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:21.770 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:21.775 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:21.779 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:21.775+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:21.786 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-19T21:36:21.797 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:21.819 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:21.845 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:21.848 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:21.879 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:21.871+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:21.927 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671376311000003, 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:21.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:21.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671362771999995, 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:21.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671337031999997, 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671322817999993, 2024-09-19T21:36:21.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:21.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671301901000007, 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:21.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:21.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671286524999999, 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.952 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671271597, 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:21.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671218976999995, 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:21.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:21.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671169399000007, 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:21.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671152183999993, 2024-09-19T21:36:21.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:21.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:21.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671133045999994, 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:21.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671118198000002, 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:21.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.671095205, 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.670362913000005, 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:21.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.670312602999999, 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:21.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.670250291000002, 2024-09-19T21:36:21.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:21.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:22.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.668919657999993, 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:22.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:22.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.668784079999995, 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.008 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.668707320999999, 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:22.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.016 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 67.668648597000001, 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.021 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.023 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:36:22.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:22.024 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:22.015+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:22.124 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:22.115+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:22.177 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.177 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:22.177 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:22.177 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:22.225 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:22.219+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:22.292 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:22.293 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:22.326 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:22.323+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:22.414 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.414 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:22.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:22.414 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:22.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.212101287000003, 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:22.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.211521376, 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:22.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.211458729, 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:22.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.211392408999998, 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.434 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210688730999998, 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.435 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:22.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.439 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210615908999998, 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:22.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210602170999998, 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:22.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210584482000002, 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210534096000004, 2024-09-19T21:36:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:22.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:22.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210507206999999, 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:22.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210466631000003, 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210445016999998, 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.210425817999997, 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:22.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.721708323999998, 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:22.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:22.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.721441046999999, 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:22.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.721108215999998, 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.487 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:22.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.720561072000002, 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:22.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:22.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.720484734000003, 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.720152458000001, 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.719849531999998, 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.511 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:22.513 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:22.507+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:22.515 INFO:teuthology.orchestra.run.smithi122.stdout:4965 write 0xd660800 thru 0xd66c82d (0xc02e bytes) 2024-09-19T21:36:22.518 INFO:teuthology.orchestra.run.smithi122.stdout:4966 write 0x3008200 thru 0x300bfc3 (0x3dc4 bytes) 2024-09-19T21:36:22.519 INFO:teuthology.orchestra.run.smithi122.stdout:4967 punch from 0x76d0600 to 0x76d6600, (0x6000 bytes) 2024-09-19T21:36:22.593 INFO:teuthology.orchestra.run.smithi122.stdout:4969 trunc from 0xd66ca00 to 0xbfee00 2024-09-19T21:36:22.612 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:22.603+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:22.626 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.627 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:22.627 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:22.627 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:22.649 INFO:teuthology.orchestra.run.smithi122.stdout:4975 flatten 2024-09-19T21:36:22.689 INFO:teuthology.orchestra.run.smithi122.stdout:4976 read 0x810c00 thru 0x815630 (0x4a31 bytes) 2024-09-19T21:36:22.692 INFO:teuthology.orchestra.run.smithi122.stdout:4977 read 0x248600 thru 0x24c8d3 (0x42d4 bytes) 2024-09-19T21:36:22.692 INFO:teuthology.orchestra.run.smithi122.stdout:4978 read 0xb6c400 thru 0xb789d4 (0xc5d5 bytes) 2024-09-19T21:36:22.693 INFO:teuthology.orchestra.run.smithi122.stdout:4979 read 0x14f800 thru 0x14f88f (0x90 bytes) 2024-09-19T21:36:22.693 INFO:teuthology.orchestra.run.smithi122.stdout:4981 read 0xa4000 thru 0xa72a0 (0x32a1 bytes) 2024-09-19T21:36:22.693 INFO:teuthology.orchestra.run.smithi122.stdout:4985 punch from 0xa3600 to 0xa8800, (0x5200 bytes) 2024-09-19T21:36:22.713 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:22.707+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:22.739 INFO:teuthology.orchestra.run.smithi122.stdout:4988 write 0xd6af600 thru 0xd6b92d8 (0x9cd9 bytes) 2024-09-19T21:36:22.746 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.746 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:22.746 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:22.746 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:22.746 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:22.755 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:22.770 INFO:teuthology.orchestra.run.smithi122.stdout:4990 write 0xd92d000 thru 0xd9351b7 (0x81b8 bytes) 2024-09-19T21:36:22.773 INFO:teuthology.orchestra.run.smithi122.stdout:4991 punch from 0x48a3800 to 0x48a6e00, (0x3600 bytes) 2024-09-19T21:36:22.774 INFO:teuthology.orchestra.run.smithi122.stdout:4995 read 0x1be3c00 thru 0x1be76fd (0x3afe bytes) 2024-09-19T21:36:22.788 INFO:teuthology.orchestra.run.smithi122.stdout:4999 trunc from 0xd935200 to 0x7a17200 2024-09-19T21:36:22.813 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:22.807+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:22.818 INFO:teuthology.orchestra.run.smithi122.stdout:5001 write 0x403d400 thru 0x40455cd (0x81ce bytes) 2024-09-19T21:36:22.830 INFO:teuthology.orchestra.run.smithi122.stdout:5003 trunc from 0x7a17200 to 0xbbfa800 2024-09-19T21:36:22.860 INFO:teuthology.orchestra.run.smithi122.stdout:5005 write 0x2808c00 thru 0x28140c5 (0xb4c6 bytes) 2024-09-19T21:36:22.861 INFO:teuthology.orchestra.run.smithi122.stdout:5007 punch from 0xadaaa00 to 0xadaea00, (0x4000 bytes) 2024-09-19T21:36:22.862 INFO:teuthology.orchestra.run.smithi122.stdout:5008 read 0xb08a000 thru 0xb093bb4 (0x9bb5 bytes) 2024-09-19T21:36:22.862 INFO:teuthology.orchestra.run.smithi122.stdout:5009 write 0x9837800 thru 0x983a057 (0x2858 bytes) 2024-09-19T21:36:22.894 INFO:teuthology.orchestra.run.smithi122.stdout:5011 write 0xc136e00 thru 0xc143322 (0xc523 bytes) 2024-09-19T21:36:22.914 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:22.911+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:22.920 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 87.167983273000004, 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:22.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540643877000001, 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540591411000001, 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:22.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540560864, 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:22.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540538267000002, 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:22.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540515814999999, 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:22.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540501519000003, 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.952 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:22.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540484892000002, 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:22.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.961 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540466256000002, 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540439591000002, 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:22.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540429379999999, 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:22.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.540416894000003, 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:22.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:22.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536542304000001, 2024-09-19T21:36:22.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:22.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536524784999997, 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:22.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536487231000002, 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:22.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536469388, 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:22.995 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536456938999997, 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:23.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536446247000001, 2024-09-19T21:36:23.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.536434669999998, 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 42.120462713999999, 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:23.018 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:23.020 INFO:teuthology.orchestra.run.smithi122.stdout:5012 write 0xd4c3e00 thru 0xd4c615b (0x235c bytes) 2024-09-19T21:36:23.020 INFO:teuthology.orchestra.run.smithi122.stdout:5014 trunc from 0xd4c6200 to 0x5fc3200 2024-09-19T21:36:23.020 INFO:teuthology.orchestra.run.smithi122.stdout:5015 trunc from 0x5fc3200 to 0x61e8200 2024-09-19T21:36:23.020 INFO:teuthology.orchestra.run.smithi122.stdout:5018 read 0x2899400 thru 0x28a1dd8 (0x89d9 bytes) 2024-09-19T21:36:23.021 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:23.021 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:23.015+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:23.053 INFO:teuthology.orchestra.run.smithi122.stdout:5019 write 0xdc14200 thru 0xdc1dc35 (0x9a36 bytes) 2024-09-19T21:36:23.056 INFO:teuthology.orchestra.run.smithi122.stdout:5021 read 0xf51000 thru 0xf58f1b (0x7f1c bytes) 2024-09-19T21:36:23.056 INFO:teuthology.orchestra.run.smithi122.stdout:5023 write 0x98f4800 thru 0x98f7546 (0x2d47 bytes) 2024-09-19T21:36:23.056 INFO:teuthology.orchestra.run.smithi122.stdout:5024 write 0x2240400 thru 0x224d71f (0xd320 bytes) 2024-09-19T21:36:23.056 INFO:teuthology.orchestra.run.smithi122.stdout:5025 read 0x942200 thru 0x951283 (0xf084 bytes) 2024-09-19T21:36:23.057 INFO:teuthology.orchestra.run.smithi122.stdout:5026 read 0x4f9aa00 thru 0x4f9d6c5 (0x2cc6 bytes) 2024-09-19T21:36:23.058 INFO:teuthology.orchestra.run.smithi122.stdout:5028 write 0x26b5000 thru 0x26b80fe (0x30ff bytes) 2024-09-19T21:36:23.058 INFO:teuthology.orchestra.run.smithi122.stdout:5029 write 0x26a9000 thru 0x26b8e05 (0xfe06 bytes) 2024-09-19T21:36:23.078 INFO:teuthology.orchestra.run.smithi122.stdout:5030 trunc from 0xdc1de00 to 0x3f3a800 2024-09-19T21:36:23.120 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:23.115+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:23.127 INFO:teuthology.orchestra.run.smithi122.stdout:5033 read 0x12be00 thru 0x136d30 (0xaf31 bytes) 2024-09-19T21:36:23.129 INFO:teuthology.orchestra.run.smithi122.stdout:5034 write 0x943600 thru 0x94eef5 (0xb8f6 bytes) 2024-09-19T21:36:23.145 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:23.145 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.155 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:23.176 INFO:teuthology.orchestra.run.smithi122.stdout:5035 write 0xd101a00 thru 0xd10736a (0x596b bytes) 2024-09-19T21:36:23.178 INFO:teuthology.orchestra.run.smithi122.stdout:5038 read 0x3059000 thru 0x3061f07 (0x8f08 bytes) 2024-09-19T21:36:23.178 INFO:teuthology.orchestra.run.smithi122.stdout:5039 write 0xabd0a00 thru 0xabd3eba (0x34bb bytes) 2024-09-19T21:36:23.178 INFO:teuthology.orchestra.run.smithi122.stdout:5042 read 0xc434800 thru 0xc43e559 (0x9d5a bytes) 2024-09-19T21:36:23.178 INFO:teuthology.orchestra.run.smithi122.stdout:5043 read 0x5f5a000 thru 0x5f6910e (0xf10f bytes) 2024-09-19T21:36:23.179 INFO:teuthology.orchestra.run.smithi122.stdout:5045 punch from 0x7fd3c00 to 0x7fdc200, (0x8600 bytes) 2024-09-19T21:36:23.180 INFO:teuthology.orchestra.run.smithi122.stdout:5046 write 0xc3c3c00 thru 0xc3c9910 (0x5d11 bytes) 2024-09-19T21:36:23.180 INFO:teuthology.orchestra.run.smithi122.stdout:5047 write 0x91b5600 thru 0x91bb2b7 (0x5cb8 bytes) 2024-09-19T21:36:23.180 INFO:teuthology.orchestra.run.smithi122.stdout:5048 clone 33 order 24 su 131072 sc 15 2024-09-19T21:36:23.222 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:23.215+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:23.266 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:23.266 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:23.267 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:23.267 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:23.323 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:23.319+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:23.387 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.026629555, 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:36:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:36:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.026617246999997, 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.392 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:23.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:36:23.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.396 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.993886007999997, 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:36:23.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:36:23.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.764128315000001, 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.402 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:36:23.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.764104879999998, 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.407 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:23.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.764083151999998, 2024-09-19T21:36:23.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:23.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.415 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.764062420000002, 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:23.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:23.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.420 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.764027984000002, 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:23.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:23.425 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.763985187000003, 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:23.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:23.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758759677, 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758737379000003, 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:23.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:23.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758706523999997, 2024-09-19T21:36:23.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:23.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:23.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758663814999998, 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.444 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758639828, 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:23.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:23.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.453 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758608424000002, 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:23.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:23.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758583258000002, 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:23.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758538180000002, 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:23.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758511435999999, 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.467 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:23.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758498144000001, 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:23.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:23.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.758481224999997, 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:23.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:23.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.479 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:23.481 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:23.475+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:23.582 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:23.575+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:23.632 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:23.632 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:23.632 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:23.686 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:23.683+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:23.753 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:23.753 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:23.754 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.765 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:23.784 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:23.779+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:23.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:23.878 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:23.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.94257531300001, 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:23.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:23.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.885 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.623007981999997, 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:23.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622996215999997, 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.890 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:23.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:23.894 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622983323, 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.898 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622971300000003, 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.899 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622953656, 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.903 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:23.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:23.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.62293786, 2024-09-19T21:36:23.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:23.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.911 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622926008999997, 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622857164999999, 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:23.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:23.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:23.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622829025000001, 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:23.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.622805896000003, 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:23.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.608029276000003, 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:23.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.600692961, 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:23.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.600672070999998, 2024-09-19T21:36:23.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.942 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.593596531000003, 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:23.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:23.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:23.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.593575987000001, 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:23.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:23.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:23.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.413221915999998, 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.951 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:23.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.413211488999998, 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:23.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.413201059999999, 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:23.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:23.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:23.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:23.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.413190997000001, 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:23.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:23.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:23.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:23.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:23.973 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:23.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:23.975 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:23.971+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:24.001 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:24.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:36:24.026 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:24.053 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:24.075 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:24.067+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:24.076 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:24.103 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:24.112 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.112 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:24.125 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:24.126 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:24.127 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:24.176 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:24.167+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:24.223 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.223 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:24.277 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:24.271+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:24.341 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.341 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:24.397 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:24.391+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:24.471 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.471 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.479 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:24.475+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:24.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:24.581 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:24.575+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:24.596 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.596 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:24.626 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4] out_osds: [5, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:24.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-19T21:36:24.682 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:24.675+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:24.713 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.713 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:24.755 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-19T21:36:24.785 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:24.779+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:24.844 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.845 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:24.886 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:24.883+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:24.965 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:24.965 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:24.973 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:36:24.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:24.989 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:24.983+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:25.090 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:25.087+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:25.091 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.091 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:25.126 INFO:teuthology.orchestra.run.smithi122.stdout:5052 read 0x6c49a00 thru 0x6c4bc21 (0x2222 bytes) 2024-09-19T21:36:25.128 INFO:teuthology.orchestra.run.smithi122.stdout:5054 read 0xb866c00 thru 0xb86c36d (0x576e bytes) 2024-09-19T21:36:25.129 INFO:teuthology.orchestra.run.smithi122.stdout:5057 read 0x9cb6200 thru 0x9cc2c41 (0xca42 bytes) 2024-09-19T21:36:25.130 INFO:teuthology.orchestra.run.smithi122.stdout:5058 write 0x74d8a00 thru 0x74dc5a2 (0x3ba3 bytes) 2024-09-19T21:36:25.131 INFO:teuthology.orchestra.run.smithi122.stdout:5059 read 0xb152e00 thru 0xb15da77 (0xac78 bytes) 2024-09-19T21:36:25.133 INFO:teuthology.orchestra.run.smithi122.stdout:5060 read 0x4dd9c00 thru 0x4de27b9 (0x8bba bytes) 2024-09-19T21:36:25.133 INFO:teuthology.orchestra.run.smithi122.stdout:5061 clone 34 order 23 su 65536 sc 6 2024-09-19T21:36:25.194 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:25.187+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:25.211 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.211 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:25.295 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:25.287+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:25.395 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:25.387+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.457 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:25.501 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:25.495+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:25.576 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.576 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:25.596 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:25.591+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:25.696 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.794 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.794 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:25.691+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:25.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:25.797 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:25.795+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:25.899 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:25.895+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:25.908 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:25.908 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:25.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:25.984 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-09-19T21:36:25.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-19T21:36:25.999 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:25.991+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:26.078 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:26.078 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:26.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:26.100 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:26.095+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:26.120 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone33 from 0xd107400 (overlap 0xd107400) to 0x0 2024-09-19T21:36:26.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:26.200 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:26.195+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:26.203 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:26.203 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:26.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:26.301 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:26.301 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:26.295+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:26.335 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 130.408328235, 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:26.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:26.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:26.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:26.351 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088760903999997, 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:26.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088749137999997, 2024-09-19T21:36:26.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:26.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:26.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088736245, 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:26.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.364 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:26.365 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088724222000003, 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.366 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.368 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:26.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088706578, 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.371 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.372 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.373 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:26.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088690782, 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:26.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.376 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:26.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088678930999997, 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:26.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:26.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088610086999999, 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.385 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.389 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088581947000002, 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.390 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:26.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.088558818000003, 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:26.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.073782198000004, 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.399 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.403 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.066445883, 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:26.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.066424992999998, 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.408 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:26.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.059349453000003, 2024-09-19T21:36:26.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:26.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.416 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.059328909000001, 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:26.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:26.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.425 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 53.878974837999998, 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:26.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 53.878964410999998, 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:26.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:26.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 53.878953981999999, 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.435 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:26.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:26.439 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 53.878943919000001, 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:26.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:26.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.444 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:26.446 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:26.446 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:26.447 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:26.447 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:26.447 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:26.448 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:26.439+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:26.546 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:26.543+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:26.560 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:26.560 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:26.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:26.648 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:26.643+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:26.734 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:26.734 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:26.734 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:26.734 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:26.734 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:26.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:26.749 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:26.747+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:26.850 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:26.843+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:26.852 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 91.120716490999996, 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.853 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:26.855 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:26.857 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.858 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518761247, 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.860 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:26.861 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.862 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.863 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5147 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 84992~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518348899999999, 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054506770000000003, 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5147 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331567+0000", 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.865 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331566+0000", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331569+0000", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7219999999999999e-06 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331575+0000", 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.609e-06 2024-09-19T21:36:26.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331579+0000", 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385792+0000", 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054212916 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.867 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.385870+0000", 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7725999999999994e-05 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386073+0000", 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020359299999999999 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.868 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5148 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 146432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518331998000001, 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054994451, 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5148 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331584+0000", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331583+0000", 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331586+0000", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4279999999999999e-06 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331592+0000", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8640000000000001e-06 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331596+0000", 2024-09-19T21:36:26.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9019999999999998e-06 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386100+0000", 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054503916999999999 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386172+0000", 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1348999999999997e-05 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386578+0000", 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00040632100000000002 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518315264999998, 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.874 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:26.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.877 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5202 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 5120~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518234212999999, 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055242990999999998, 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5202 2024-09-19T21:36:26.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.879 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331681+0000", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331683+0000", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3439999999999999e-06 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331689+0000", 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6890000000000001e-06 2024-09-19T21:36:26.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331694+0000", 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8010000000000003e-06 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386602+0000", 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.054908449999999998 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0025000000000006e-05 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386924+0000", 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023204700000000001 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.882 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5203 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 65536~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518217483000001, 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055520511000000002, 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5203 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331698+0000", 2024-09-19T21:36:26.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331700+0000", 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5270000000000001e-06 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331706+0000", 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1880000000000001e-06 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331711+0000", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0730000000000004e-06 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386947+0000", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055236889999999997 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387013+0000", 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5092000000000006e-05 2024-09-19T21:36:26.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387219+0000", 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020603599999999999 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518201085999998, 2024-09-19T21:36:26.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:26.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518184652999999, 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:26.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:26.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518169096999998, 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:26.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.901 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518055027999999, 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.902 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:26.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:26.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.906 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518038064999999, 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:26.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.911 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518022080999998, 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:26.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:26.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:26.916 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.518004906000002, 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:26.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:26.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517987708, 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:26.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:26.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.925 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517970877000003, 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:26.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.930 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517954355999997, 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:26.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517931752000003, 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:26.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:26.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517915363999997, 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.517806483000001, 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:26.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:26.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:26.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:26.948 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:26.949 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-19T21:36:26.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:26.951 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:26.947+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:26.985 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:36:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-09-19T21:36:27.010 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-19T21:36:27.054 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:27.051+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:27.084 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.084 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:27.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.084 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5492.0:1685 3.18 3.93746f58 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:27.082375+0000", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 5.1189000000000001e-05, 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8905000000000003e-05, 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5492", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3718535322", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1685 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082375+0000", 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082375+0000", 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082373+0000", 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082382+0000", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9530000000000005e-06 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082406+0000", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3530999999999999e-05 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082426+0000", 2024-09-19T21:36:27.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.054e-05 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082478+0000", 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1257000000000001e-05 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5492.0:1686 3.18 3.93746f58 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-19T21:36:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:27.082471+0000", 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4294967295.9999557, 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015459499999999999, 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5492", 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3718535322", 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1686 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082471+0000", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082471+0000", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082469+0000", 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082477+0000", 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.391e-06 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082491+0000", 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4544e-05 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082502+0000", 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0331e-05 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082597+0000", 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5047e-05 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5492.0:1689 3.5 3:a048aae8:::rbd_data.11064211bd69.00000000000000cf:head [sparse-read 135168~126976] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-19T21:36:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:27.082517+0000", 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4294967295.9999089, 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000240463, 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "reached pg", 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5492", 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3718535322", 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1689 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082517+0000", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082517+0000", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082516+0000", 2024-09-19T21:36:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082521+0000", 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0839999999999999e-06 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082534+0000", 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2724999999999999e-05 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082544+0000", 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.533e-06 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082607+0000", 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3113000000000004e-05 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5492.0:1690 3.5 3.17551205 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:27.082567+0000", 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4294967295.9998593, 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000322388, 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "queued for pg", 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5492", 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3718535322", 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1690 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082567+0000", 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082567+0000", 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082566+0000", 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082570+0000", 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.7180000000000004e-06 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082727+0000", 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000156627 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:27.082739+0000", 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.224e-05 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 4 2024-09-19T21:36:27.100 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:27.153 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:27.147+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:27.218 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:27.218 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:27.218 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:27.218 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:27.254 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:27.251+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:27.336 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078158635999998, 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:27.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078145097000004, 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:27.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.345 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078119357000006, 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:27.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078105143000002, 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078084226000001, 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.355 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:27.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.359 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078068849999994, 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078053921999995, 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:27.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.078001302000004, 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:27.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:27.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077951724000002, 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:27.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:27.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077934509000002, 2024-09-19T21:36:27.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:27.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:27.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077915371000003, 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:27.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077900522999997, 2024-09-19T21:36:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077877529999995, 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.391 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:27.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077145238, 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:27.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:27.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077094927999994, 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:27.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:27.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.077032615999997, 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.075701983000002, 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:27.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.075566405000004, 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:27.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.075489645999994, 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.418 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:27.422 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.075430921999995, 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.427 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:27.429 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:27.423+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:27.468 INFO:teuthology.orchestra.run.smithi122.stdout:5063 write 0xc76fa00 thru 0xc77804e (0x864f bytes) 2024-09-19T21:36:27.470 INFO:teuthology.orchestra.run.smithi122.stdout:5064 write 0x8227e00 thru 0x8236b96 (0xed97 bytes) 2024-09-19T21:36:27.472 INFO:teuthology.orchestra.run.smithi122.stdout:5065 write 0x2f83c00 thru 0x2f8b027 (0x7428 bytes) 2024-09-19T21:36:27.474 INFO:teuthology.orchestra.run.smithi122.stdout:5066 punch from 0x4f99000 to 0x4fa5e00, (0xce00 bytes) 2024-09-19T21:36:27.529 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:27.523+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:27.537 INFO:teuthology.orchestra.run.smithi122.stdout:5068 read 0x5c3ae00 thru 0x5c48fa1 (0xe1a2 bytes) 2024-09-19T21:36:27.538 INFO:teuthology.orchestra.run.smithi122.stdout:5069 write 0xbd7e200 thru 0xbd8bcba (0xdabb bytes) 2024-09-19T21:36:27.539 INFO:teuthology.orchestra.run.smithi122.stdout:5072 punch from 0x56c0600 to 0x56c5c00, (0x5600 bytes) 2024-09-19T21:36:27.577 INFO:teuthology.orchestra.run.smithi122.stdout:5073 read 0x836d000 thru 0x8377d79 (0xad7a bytes) 2024-09-19T21:36:27.578 INFO:teuthology.orchestra.run.smithi122.stdout:5074 punch from 0x6291600 to 0x62a0600, (0xf000 bytes) 2024-09-19T21:36:27.592 INFO:teuthology.orchestra.run.smithi122.stdout:5076 write 0x9b3c200 thru 0x9b40e09 (0x4c0a bytes) 2024-09-19T21:36:27.592 INFO:teuthology.orchestra.run.smithi122.stdout:5077 clone 35 order 18 su 65536 sc 4 2024-09-19T21:36:27.593 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.593 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:27.593 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:27.593 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:27.630 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:27.627+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:27.707 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:27.708 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.718 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:27.730 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:27.727+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:27.832 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:27.827+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:27.876 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.674532673000002, 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673952761999999, 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673890114999999, 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673823794999997, 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673120117000003, 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673047295000003, 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673033556999997, 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.673015868, 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.672965482000002, 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.672938592999998, 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.672898017000001, 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.672876402999997, 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.672857204000003, 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.184139709999997, 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.183872432999998, 2024-09-19T21:36:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.183539602000003, 2024-09-19T21:36:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.182992458000001, 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.182916120000002, 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.182583844, 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.182280917999996, 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:27.970 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:27.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:27.972 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:27.971+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:36:28.009 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-09-19T21:36:28.023 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-19T21:36:28.072 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:28.067+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:28.091 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:28.091 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:28.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:28.169 INFO:teuthology.orchestra.run.smithi122.stdout:truncating image image_client.0-clone34 from 0xd107400 (overlap 0xd107400) to 0x29ce400 2024-09-19T21:36:28.172 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:28.167+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:28.273 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:28.271+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:28.286 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:28.287 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:28.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:28.374 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:28.371+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.659014576999994, 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031675180999997, 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031622714999997, 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031592168000003, 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031569570999999, 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031547119000003, 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031532822999999, 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031516195999998, 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031497559999998, 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031470894999998, 2024-09-19T21:36:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031460684000002, 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.031448198, 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027573607999997, 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027556089000001, 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027518534999999, 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027500691999997, 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027488243000001, 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027477550999997, 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 48.027465974000002, 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 47.611494018000002, 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:28.508 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:28.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:28.510 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:28.507+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:28.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:28.611 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:28.607+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:28.623 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:28.623 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:28.623 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:28.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:28.660 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:28.681 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:28.703 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:28.712 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:28.707+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:28.725 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:28.738 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:28.738 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:28.746 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:28.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:28.771 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:28.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:28.813 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:28.811+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:28.854 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:28.854 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:28.865 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:28.914 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:28.911+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:28.962 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-19T21:36:28.977 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:28.977 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:28.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:29.010 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-09-19T21:36:29.014 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:29.011+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:29.022 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-19T21:36:29.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:36:29.081 INFO:teuthology.orchestra.run.smithi122.stdout:5079 trunc from 0xd107400 to 0xb94de00 2024-09-19T21:36:29.104 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.104 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:29.116 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:29.111+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:29.121 INFO:teuthology.orchestra.run.smithi122.stdout:5081 read 0x8bcee00 thru 0x8bd8115 (0x9316 bytes) 2024-09-19T21:36:29.123 INFO:teuthology.orchestra.run.smithi122.stdout:5082 write 0x68ab800 thru 0x68b866b (0xce6c bytes) 2024-09-19T21:36:29.125 INFO:teuthology.orchestra.run.smithi122.stdout:5083 read 0x641fc00 thru 0x64250fd (0x54fe bytes) 2024-09-19T21:36:29.126 INFO:teuthology.orchestra.run.smithi122.stdout:5084 write 0x7ea3200 thru 0x7eaf7bc (0xc5bd bytes) 2024-09-19T21:36:29.172 INFO:teuthology.orchestra.run.smithi122.stdout:5085 write 0xe89f800 thru 0xe89fccc (0x4cd bytes) 2024-09-19T21:36:29.174 INFO:teuthology.orchestra.run.smithi122.stdout:5086 write 0x5b5ca00 thru 0x5b602c9 (0x38ca bytes) 2024-09-19T21:36:29.174 INFO:teuthology.orchestra.run.smithi122.stdout:5089 punch from 0x9c9400 to 0x9d2400, (0x9000 bytes) 2024-09-19T21:36:29.176 INFO:teuthology.orchestra.run.smithi122.stdout:5090 write 0xbe81e00 thru 0xbe82b8d (0xd8e bytes) 2024-09-19T21:36:29.176 INFO:teuthology.orchestra.run.smithi122.stdout:5091 write 0x6611e00 thru 0x66126cb (0x8cc bytes) 2024-09-19T21:36:29.178 INFO:teuthology.orchestra.run.smithi122.stdout:5092 punch from 0x990fc00 to 0x9915400, (0x5800 bytes) 2024-09-19T21:36:29.216 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:29.211+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:29.223 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.223 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:5093 trunc from 0xe89fe00 to 0x24bdc00 2024-09-19T21:36:29.317 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:29.311+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:29.341 INFO:teuthology.orchestra.run.smithi122.stdout:5095 write 0xe6eb600 thru 0xe6ece51 (0x1852 bytes) 2024-09-19T21:36:29.342 INFO:teuthology.orchestra.run.smithi122.stdout:5096 write 0x83a800 thru 0x8418dc (0x70dd bytes) 2024-09-19T21:36:29.346 INFO:teuthology.orchestra.run.smithi122.stdout:5097 write 0x9777a00 thru 0x977af86 (0x3587 bytes) 2024-09-19T21:36:29.347 INFO:teuthology.orchestra.run.smithi122.stdout:5100 punch from 0x43d4600 to 0x43dc800, (0x8200 bytes) 2024-09-19T21:36:29.348 INFO:teuthology.orchestra.run.smithi122.stdout:5108 write 0x2126a00 thru 0x212a1cb (0x37cc bytes) 2024-09-19T21:36:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.349 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.353 INFO:teuthology.orchestra.run.smithi122.stdout:5113 read 0x7d27000 thru 0x7d30b82 (0x9b83 bytes) 2024-09-19T21:36:29.354 INFO:teuthology.orchestra.run.smithi122.stdout:5114 read 0xd7ecc00 thru 0xd7ee470 (0x1871 bytes) 2024-09-19T21:36:29.354 INFO:teuthology.orchestra.run.smithi122.stdout:5115 write 0x9b6ec00 thru 0x9b78dc4 (0xa1c5 bytes) 2024-09-19T21:36:29.354 INFO:teuthology.orchestra.run.smithi122.stdout:5116 write 0x904fe00 thru 0x90525cb (0x27cc bytes) 2024-09-19T21:36:29.354 INFO:teuthology.orchestra.run.smithi122.stdout:5118 write 0xb0ad600 thru 0xb0ada02 (0x403 bytes) 2024-09-19T21:36:29.355 INFO:teuthology.orchestra.run.smithi122.stdout:5121 punch from 0xaf44000 to 0xaf44400, (0x400 bytes) 2024-09-19T21:36:29.355 INFO:teuthology.orchestra.run.smithi122.stdout:5122 read 0x3f24a00 thru 0x3f288aa (0x3eab bytes) 2024-09-19T21:36:29.356 INFO:teuthology.orchestra.run.smithi122.stdout:5123 write 0x1410400 thru 0x141ee0f (0xea10 bytes) 2024-09-19T21:36:29.357 INFO:teuthology.orchestra.run.smithi122.stdout:5125 read 0x4e1a600 thru 0x4e24832 (0xa233 bytes) 2024-09-19T21:36:29.357 INFO:teuthology.orchestra.run.smithi122.stdout:5127 write 0xc349200 thru 0xc357135 (0xdf36 bytes) 2024-09-19T21:36:29.358 INFO:teuthology.orchestra.run.smithi122.stdout:5128 write 0x6f63400 thru 0x6f701bf (0xcdc0 bytes) 2024-09-19T21:36:29.358 INFO:teuthology.orchestra.run.smithi122.stdout:5129 punch from 0x230b800 to 0x230c400, (0xc00 bytes) 2024-09-19T21:36:29.360 INFO:teuthology.orchestra.run.smithi122.stdout:5130 write 0x18fa200 thru 0x18ff819 (0x561a bytes) 2024-09-19T21:36:29.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:29.384 INFO:teuthology.orchestra.run.smithi122.stdout:5133 trunc from 0xe6ed000 to 0x33fe400 2024-09-19T21:36:29.407 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:36:29.418 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:29.411+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:29.421 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-19T21:36:29.001257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1877731177,"num_objects":1868,"num_object_clones":528,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":719,"num_read":260491,"num_read_kb":878955,"num_write":13744,"num_write_kb":228269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1681,"num_bytes_recovered":110437112,"num_keys_recovered":253,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12902,"ondisk_log_size":12902,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":503836,"kb_used_data":326880,"kb_used_omap":6,"kb_used_meta":176889,"kb_avail":376983524,"statfs":{"total":386547056640,"available":386031128576,"internally_reserved":0,"allocated":334725120,"data_stored":424694876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":181135130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":116444189,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":28,"num_read":5244,"num_read_kb":7616,"num_write":265,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032525"},"pg_stats":[{"pgid":"3.1f","version":"94'319","reported_seq":3150,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.993021+0000","last_change":"2024-09-19T21:36:06.849962+0000","last_active":"2024-09-19T21:36:26.993021+0000","last_peered":"2024-09-19T21:36:26.993021+0000","last_clean":"2024-09-19T21:36:26.993021+0000","last_became_active":"2024-09-19T21:36:06.848734+0000","last_became_peered":"2024-09-19T21:36:06.848734+0000","last_unstale":"2024-09-19T21:36:26.993021+0000","last_undegraded":"2024-09-19T21:36:26.993021+0000","last_fullsized":"2024-09-19T21:36:26.993021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:12.838002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":44079749,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":17,"num_read":4356,"num_read_kb":35957,"num_write":322,"num_write_kb":22708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":24095839,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"97'207","reported_seq":1676,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.267548+0000","last_change":"2024-09-19T21:36:17.831424+0000","last_active":"2024-09-19T21:36:26.267548+0000","last_peered":"2024-09-19T21:36:26.267548+0000","last_clean":"2024-09-19T21:36:26.267548+0000","last_became_active":"2024-09-19T21:36:17.830193+0000","last_became_peered":"2024-09-19T21:36:17.830193+0000","last_unstale":"2024-09-19T21:36:26.267548+0000","last_undegraded":"2024-09-19T21:36:26.267548+0000","last_fullsized":"2024-09-19T21:36:26.267548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:32:51.816667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":33576004,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":1268,"num_read_kb":6951,"num_write":159,"num_write_kb":3211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1525316,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"99'1015","reported_seq":7216,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.144038+0000","last_change":"2024-09-19T21:36:18.837912+0000","last_active":"2024-09-19T21:36:28.144038+0000","last_peered":"2024-09-19T21:36:28.144038+0000","last_clean":"2024-09-19T21:36:28.144038+0000","last_became_active":"2024-09-19T21:36:18.836618+0000","last_became_peered":"2024-09-19T21:36:18.836618+0000","last_unstale":"2024-09-19T21:36:28.144038+0000","last_undegraded":"2024-09-19T21:36:28.144038+0000","last_fullsized":"2024-09-19T21:36:28.144038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:17:07.164147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":61966273,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":12637,"num_read_kb":31246,"num_write":1297,"num_write_kb":14357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":11559117,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"99'416","reported_seq":15555,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.084577+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:36:28.084577+0000","last_peered":"2024-09-19T21:36:28.084577+0000","last_clean":"2024-09-19T21:36:28.084577+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:36:28.084577+0000","last_undegraded":"2024-09-19T21:36:28.084577+0000","last_fullsized":"2024-09-19T21:36:28.084577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":43957300,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":16398,"num_read_kb":25708,"num_write":342,"num_write_kb":3883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"97'313","reported_seq":4017,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.443905+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:36:26.443905+0000","last_peered":"2024-09-19T21:36:26.443905+0000","last_clean":"2024-09-19T21:36:26.443905+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:36:26.443905+0000","last_undegraded":"2024-09-19T21:36:26.443905+0000","last_fullsized":"2024-09-19T21:36:26.443905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":177769161,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":3908,"num_read_kb":25123,"num_write":303,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"99'789","reported_seq":7303,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.262502+0000","last_change":"2024-09-19T21:36:05.849867+0000","last_active":"2024-09-19T21:36:28.262502+0000","last_peered":"2024-09-19T21:36:28.262502+0000","last_clean":"2024-09-19T21:36:28.262502+0000","last_became_active":"2024-09-19T21:36:05.848564+0000","last_became_peered":"2024-09-19T21:36:05.848564+0000","last_unstale":"2024-09-19T21:36:28.262502+0000","last_undegraded":"2024-09-19T21:36:28.262502+0000","last_fullsized":"2024-09-19T21:36:28.262502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.523212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":49090202,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":19,"num_read":10044,"num_read_kb":36348,"num_write":1221,"num_write_kb":8141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4294810,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"99'329","reported_seq":14020,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.247790+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:36:28.247790+0000","last_peered":"2024-09-19T21:36:28.247790+0000","last_clean":"2024-09-19T21:36:28.247790+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:36:28.247790+0000","last_undegraded":"2024-09-19T21:36:28.247790+0000","last_fullsized":"2024-09-19T21:36:28.247790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":39406786,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":20,"num_read":14484,"num_read_kb":30926,"num_write":413,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"97'228","reported_seq":8167,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.278339+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:36:26.278339+0000","last_peered":"2024-09-19T21:36:26.278339+0000","last_clean":"2024-09-19T21:36:26.278339+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:36:26.278339+0000","last_undegraded":"2024-09-19T21:36:26.278339+0000","last_fullsized":"2024-09-19T21:36:26.278339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":75472484,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":8720,"num_read_kb":26159,"num_write":243,"num_write_kb":9085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"97'220","reported_seq":5651,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.241840+0000","last_change":"2024-09-19T21:36:05.848894+0000","last_active":"2024-09-19T21:36:28.241840+0000","last_peered":"2024-09-19T21:36:28.241840+0000","last_clean":"2024-09-19T21:36:28.241840+0000","last_became_active":"2024-09-19T21:36:05.848182+0000","last_became_peered":"2024-09-19T21:36:05.848182+0000","last_unstale":"2024-09-19T21:36:28.241840+0000","last_undegraded":"2024-09-19T21:36:28.241840+0000","last_fullsized":"2024-09-19T21:36:28.241840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:18.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":47604736,"num_objects":71,"num_object_clones":26,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":32,"num_read":5509,"num_read_kb":11213,"num_write":178,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"97'256","reported_seq":10358,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.782898+0000","last_change":"2024-09-19T21:36:17.832459+0000","last_active":"2024-09-19T21:36:26.782898+0000","last_peered":"2024-09-19T21:36:26.782898+0000","last_clean":"2024-09-19T21:36:26.782898+0000","last_became_active":"2024-09-19T21:36:17.831620+0000","last_became_peered":"2024-09-19T21:36:17.831620+0000","last_unstale":"2024-09-19T21:36:26.782898+0000","last_undegraded":"2024-09-19T21:36:26.782898+0000","last_fullsized":"2024-09-19T21:36:26.782898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:38:39.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":74689064,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":11271,"num_read_kb":28621,"num_write":210,"num_write_kb":5650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4500008,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"94'273","reported_seq":6135,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.485949+0000","last_change":"2024-09-19T21:36:05.846800+0000","last_active":"2024-09-19T21:36:27.485949+0000","last_peered":"2024-09-19T21:36:27.485949+0000","last_clean":"2024-09-19T21:36:27.485949+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:36:27.485949+0000","last_undegraded":"2024-09-19T21:36:27.485949+0000","last_fullsized":"2024-09-19T21:36:27.485949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":14478627,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":5653,"num_read_kb":10209,"num_write":223,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"98'250","reported_seq":9310,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.251000+0000","last_change":"2024-09-19T21:36:17.833402+0000","last_active":"2024-09-19T21:36:28.251000+0000","last_peered":"2024-09-19T21:36:28.251000+0000","last_clean":"2024-09-19T21:36:28.251000+0000","last_became_active":"2024-09-19T21:36:17.832672+0000","last_became_peered":"2024-09-19T21:36:17.832672+0000","last_unstale":"2024-09-19T21:36:28.251000+0000","last_undegraded":"2024-09-19T21:36:28.251000+0000","last_fullsized":"2024-09-19T21:36:28.251000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:17:15.919793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":49711632,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":9412,"num_read_kb":25211,"num_write":211,"num_write_kb":4367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":3118608,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.993117+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:36:26.993117+0000","last_peered":"2024-09-19T21:36:26.993117+0000","last_clean":"2024-09-19T21:36:26.993117+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:36:26.993117+0000","last_undegraded":"2024-09-19T21:36:26.993117+0000","last_fullsized":"2024-09-19T21:36:26.993117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"96'395","reported_seq":8079,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989922+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:36:26.989922+0000","last_peered":"2024-09-19T21:36:26.989922+0000","last_clean":"2024-09-19T21:36:26.989922+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:36:26.989922+0000","last_undegraded":"2024-09-19T21:36:26.989922+0000","last_fullsized":"2024-09-19T21:36:26.989922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":70333570,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":8422,"num_read_kb":21078,"num_write":351,"num_write_kb":5762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989966+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:36:26.989966+0000","last_peered":"2024-09-19T21:36:26.989966+0000","last_clean":"2024-09-19T21:36:26.989966+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:36:26.989966+0000","last_undegraded":"2024-09-19T21:36:26.989966+0000","last_fullsized":"2024-09-19T21:36:26.989966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"97'570","reported_seq":5317,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.098936+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:36:28.098936+0000","last_peered":"2024-09-19T21:36:28.098936+0000","last_clean":"2024-09-19T21:36:28.098936+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:36:28.098936+0000","last_undegraded":"2024-09-19T21:36:28.098936+0000","last_fullsized":"2024-09-19T21:36:28.098936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":25442554,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":6646,"num_read_kb":13280,"num_write":633,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.002562+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:36:28.002562+0000","last_peered":"2024-09-19T21:36:28.002562+0000","last_clean":"2024-09-19T21:36:28.002562+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:36:28.002562+0000","last_undegraded":"2024-09-19T21:36:28.002562+0000","last_fullsized":"2024-09-19T21:36:28.002562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"98'296","reported_seq":10007,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.504328+0000","last_change":"2024-09-19T21:36:18.835965+0000","last_active":"2024-09-19T21:36:27.504328+0000","last_peered":"2024-09-19T21:36:27.504328+0000","last_clean":"2024-09-19T21:36:27.504328+0000","last_became_active":"2024-09-19T21:36:18.834188+0000","last_became_peered":"2024-09-19T21:36:18.834188+0000","last_unstale":"2024-09-19T21:36:27.504328+0000","last_undegraded":"2024-09-19T21:36:27.504328+0000","last_fullsized":"2024-09-19T21:36:27.504328+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:19:08.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":75759691,"num_objects":70,"num_object_clones":23,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":9404,"num_read_kb":38414,"num_write":265,"num_write_kb":6829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":9899594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":99,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-19T21:36:24.972046+0000","last_change":"2024-09-19T21:36:14.898141+0000","last_active":"2024-09-19T21:36:24.972046+0000","last_peered":"2024-09-19T21:36:24.972046+0000","last_clean":"2024-09-19T21:36:24.972046+0000","last_became_active":"2024-09-19T21:36:14.897721+0000","last_became_peered":"2024-09-19T21:36:14.897721+0000","last_unstale":"2024-09-19T21:36:24.972046+0000","last_undegraded":"2024-09-19T21:36:24.972046+0000","last_fullsized":"2024-09-19T21:36:24.972046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:01:42.756456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"94'754","reported_seq":4872,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-19T21:36:24.971887+0000","last_change":"2024-09-19T21:36:17.832475+0000","last_active":"2024-09-19T21:36:24.971887+0000","last_peered":"2024-09-19T21:36:24.971887+0000","last_clean":"2024-09-19T21:36:24.971887+0000","last_became_active":"2024-09-19T21:36:17.831123+0000","last_became_peered":"2024-09-19T21:36:17.831123+0000","last_unstale":"2024-09-19T21:36:24.971887+0000","last_undegraded":"2024-09-19T21:36:24.971887+0000","last_fullsized":"2024-09-19T21:36:24.971887+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:17:29.475533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":40933084,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":7840,"num_read_kb":22219,"num_write":978,"num_write_kb":7884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3776220,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"98'671","reported_seq":8299,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.610338+0000","last_change":"2024-09-19T21:36:17.835218+0000","last_active":"2024-09-19T21:36:27.610338+0000","last_peered":"2024-09-19T21:36:27.610338+0000","last_clean":"2024-09-19T21:36:27.610338+0000","last_became_active":"2024-09-19T21:36:17.834370+0000","last_became_peered":"2024-09-19T21:36:17.834370+0000","last_unstale":"2024-09-19T21:36:27.610338+0000","last_undegraded":"2024-09-19T21:36:27.610338+0000","last_fullsized":"2024-09-19T21:36:27.610338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:56.289962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":59287845,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":12301,"num_read_kb":17351,"num_write":730,"num_write_kb":3977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2192165,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":121,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006803+0000","last_change":"2024-09-19T21:36:02.839832+0000","last_active":"2024-09-19T21:36:28.006803+0000","last_peered":"2024-09-19T21:36:28.006803+0000","last_clean":"2024-09-19T21:36:28.006803+0000","last_became_active":"2024-09-19T21:36:02.830895+0000","last_became_peered":"2024-09-19T21:36:02.830895+0000","last_unstale":"2024-09-19T21:36:28.006803+0000","last_undegraded":"2024-09-19T21:36:28.006803+0000","last_fullsized":"2024-09-19T21:36:28.006803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:40.111547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"99'422","reported_seq":11062,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.205350+0000","last_change":"2024-09-19T21:36:17.834758+0000","last_active":"2024-09-19T21:36:28.205350+0000","last_peered":"2024-09-19T21:36:28.205350+0000","last_clean":"2024-09-19T21:36:28.205350+0000","last_became_active":"2024-09-19T21:36:17.832993+0000","last_became_peered":"2024-09-19T21:36:17.832993+0000","last_unstale":"2024-09-19T21:36:28.205350+0000","last_undegraded":"2024-09-19T21:36:28.205350+0000","last_fullsized":"2024-09-19T21:36:28.205350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:21:49.565848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":27430505,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":12256,"num_read_kb":42540,"num_write":378,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":8857810,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:25.984013+0000","last_change":"2024-09-19T21:36:02.831334+0000","last_active":"2024-09-19T21:36:25.984013+0000","last_peered":"2024-09-19T21:36:25.984013+0000","last_clean":"2024-09-19T21:36:25.984013+0000","last_became_active":"2024-09-19T21:36:02.830988+0000","last_became_peered":"2024-09-19T21:36:02.830988+0000","last_unstale":"2024-09-19T21:36:25.984013+0000","last_undegraded":"2024-09-19T21:36:25.984013+0000","last_fullsized":"2024-09-19T21:36:25.984013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:09:35.907712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"99'394","reported_seq":7223,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.246416+0000","last_change":"2024-09-19T21:36:17.836175+0000","last_active":"2024-09-19T21:36:28.246416+0000","last_peered":"2024-09-19T21:36:28.246416+0000","last_clean":"2024-09-19T21:36:28.246416+0000","last_became_active":"2024-09-19T21:36:17.835492+0000","last_became_peered":"2024-09-19T21:36:17.835492+0000","last_unstale":"2024-09-19T21:36:28.246416+0000","last_undegraded":"2024-09-19T21:36:28.246416+0000","last_fullsized":"2024-09-19T21:36:28.246416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:56:59.854746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":83637797,"num_objects":68,"num_object_clones":24,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":27,"num_read":7643,"num_read_kb":30579,"num_write":488,"num_write_kb":9375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":5649957,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989496+0000","last_change":"2024-09-19T21:36:14.898393+0000","last_active":"2024-09-19T21:36:26.989496+0000","last_peered":"2024-09-19T21:36:26.989496+0000","last_clean":"2024-09-19T21:36:26.989496+0000","last_became_active":"2024-09-19T21:36:14.897906+0000","last_became_peered":"2024-09-19T21:36:14.897906+0000","last_unstale":"2024-09-19T21:36:26.989496+0000","last_undegraded":"2024-09-19T21:36:26.989496+0000","last_fullsized":"2024-09-19T21:36:26.989496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:39:57.143853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"97'350","reported_seq":10638,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.232391+0000","last_change":"2024-09-19T21:36:17.832488+0000","last_active":"2024-09-19T21:36:26.232391+0000","last_peered":"2024-09-19T21:36:26.232391+0000","last_clean":"2024-09-19T21:36:26.232391+0000","last_became_active":"2024-09-19T21:36:17.831369+0000","last_became_peered":"2024-09-19T21:36:17.831369+0000","last_unstale":"2024-09-19T21:36:26.232391+0000","last_undegraded":"2024-09-19T21:36:26.232391+0000","last_fullsized":"2024-09-19T21:36:26.232391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:18:45.352469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":124156439,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":17,"num_read":11722,"num_read_kb":37195,"num_write":369,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":1162263,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":108,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.002183+0000","last_change":"2024-09-19T21:36:14.906859+0000","last_active":"2024-09-19T21:36:28.002183+0000","last_peered":"2024-09-19T21:36:28.002183+0000","last_clean":"2024-09-19T21:36:28.002183+0000","last_became_active":"2024-09-19T21:36:14.898975+0000","last_became_peered":"2024-09-19T21:36:14.898975+0000","last_unstale":"2024-09-19T21:36:28.002183+0000","last_undegraded":"2024-09-19T21:36:28.002183+0000","last_fullsized":"2024-09-19T21:36:28.002183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:46:14.540137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":285,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.992571+0000","last_change":"2024-09-19T21:36:17.834100+0000","last_active":"2024-09-19T21:36:26.992571+0000","last_peered":"2024-09-19T21:36:26.992571+0000","last_clean":"2024-09-19T21:36:26.992571+0000","last_became_active":"2024-09-19T21:36:17.833564+0000","last_became_peered":"2024-09-19T21:36:17.833564+0000","last_unstale":"2024-09-19T21:36:26.992571+0000","last_undegraded":"2024-09-19T21:36:26.992571+0000","last_fullsized":"2024-09-19T21:36:26.992571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:10.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"99'391","reported_seq":14667,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.202207+0000","last_change":"2024-09-19T21:36:18.835818+0000","last_active":"2024-09-19T21:36:28.202207+0000","last_peered":"2024-09-19T21:36:28.202207+0000","last_clean":"2024-09-19T21:36:28.202207+0000","last_became_active":"2024-09-19T21:36:18.834188+0000","last_became_peered":"2024-09-19T21:36:18.834188+0000","last_unstale":"2024-09-19T21:36:28.202207+0000","last_undegraded":"2024-09-19T21:36:28.202207+0000","last_fullsized":"2024-09-19T21:36:28.202207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:17.296547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":104853592,"num_objects":86,"num_object_clones":25,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":32,"num_read":13985,"num_read_kb":55282,"num_write":414,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":7192152,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"99'349","reported_seq":5153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006192+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:36:28.006192+0000","last_peered":"2024-09-19T21:36:28.006192+0000","last_clean":"2024-09-19T21:36:28.006192+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:36:28.006192+0000","last_undegraded":"2024-09-19T21:36:28.006192+0000","last_fullsized":"2024-09-19T21:36:28.006192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":102157039,"num_objects":57,"num_object_clones":19,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":5570,"num_read_kb":16754,"num_write":322,"num_write_kb":6319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"99'462","reported_seq":7533,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006623+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:36:28.006623+0000","last_peered":"2024-09-19T21:36:28.006623+0000","last_clean":"2024-09-19T21:36:28.006623+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:36:28.006623+0000","last_undegraded":"2024-09-19T21:36:28.006623+0000","last_fullsized":"2024-09-19T21:36:28.006623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":35885876,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":27,"num_read":8961,"num_read_kb":42656,"num_write":553,"num_write_kb":12839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'190","reported_seq":2009,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.130183+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:36:26.130183+0000","last_peered":"2024-09-19T21:36:26.130183+0000","last_clean":"2024-09-19T21:36:26.130183+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:36:26.130183+0000","last_undegraded":"2024-09-19T21:36:26.130183+0000","last_fullsized":"2024-09-19T21:36:26.130183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":22110808,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":1860,"num_read_kb":19504,"num_write":182,"num_write_kb":6102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"99'172","reported_seq":2459,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.240335+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:36:28.240335+0000","last_peered":"2024-09-19T21:36:28.240335+0000","last_clean":"2024-09-19T21:36:28.240335+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:36:28.240335+0000","last_undegraded":"2024-09-19T21:36:28.240335+0000","last_fullsized":"2024-09-19T21:36:28.240335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":8029184,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":2022,"num_read_kb":6752,"num_write":134,"num_write_kb":2508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"99'332","reported_seq":8321,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.253273+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:36:28.253273+0000","last_peered":"2024-09-19T21:36:28.253273+0000","last_clean":"2024-09-19T21:36:28.253273+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:36:28.253273+0000","last_undegraded":"2024-09-19T21:36:28.253273+0000","last_fullsized":"2024-09-19T21:36:28.253273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":115963994,"num_objects":77,"num_object_clones":27,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":31,"num_read":8248,"num_read_kb":19476,"num_write":351,"num_write_kb":9155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"94'489","reported_seq":5751,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989537+0000","last_change":"2024-09-19T21:36:17.840819+0000","last_active":"2024-09-19T21:36:26.989537+0000","last_peered":"2024-09-19T21:36:26.989537+0000","last_clean":"2024-09-19T21:36:26.989537+0000","last_became_active":"2024-09-19T21:36:17.839929+0000","last_became_peered":"2024-09-19T21:36:17.839929+0000","last_unstale":"2024-09-19T21:36:26.989537+0000","last_undegraded":"2024-09-19T21:36:26.989537+0000","last_fullsized":"2024-09-19T21:36:26.989537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.623621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":33044532,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":7868,"num_read_kb":50770,"num_write":415,"num_write_kb":6304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":1322036,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"99'684","reported_seq":3984,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.227838+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:36:28.227838+0000","last_peered":"2024-09-19T21:36:28.227838+0000","last_clean":"2024-09-19T21:36:28.227838+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:36:28.227838+0000","last_undegraded":"2024-09-19T21:36:28.227838+0000","last_fullsized":"2024-09-19T21:36:28.227838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":10553711,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":6510,"num_read_kb":9984,"num_write":777,"num_write_kb":2774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"97'469","reported_seq":7224,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.483206+0000","last_change":"2024-09-19T21:36:05.849347+0000","last_active":"2024-09-19T21:36:27.483206+0000","last_peered":"2024-09-19T21:36:27.483206+0000","last_clean":"2024-09-19T21:36:27.483206+0000","last_became_active":"2024-09-19T21:36:05.848208+0000","last_became_peered":"2024-09-19T21:36:05.848208+0000","last_unstale":"2024-09-19T21:36:27.483206+0000","last_undegraded":"2024-09-19T21:36:27.483206+0000","last_fullsized":"2024-09-19T21:36:27.483206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:25:57.433432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":44381833,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":35,"num_read":8937,"num_read_kb":50131,"num_write":419,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":13819481,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"99'275","reported_seq":3156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.184473+0000","last_change":"2024-09-19T21:36:17.832573+0000","last_active":"2024-09-19T21:36:28.184473+0000","last_peered":"2024-09-19T21:36:28.184473+0000","last_clean":"2024-09-19T21:36:28.184473+0000","last_became_active":"2024-09-19T21:36:17.831693+0000","last_became_peered":"2024-09-19T21:36:17.831693+0000","last_unstale":"2024-09-19T21:36:28.184473+0000","last_undegraded":"2024-09-19T21:36:28.184473+0000","last_fullsized":"2024-09-19T21:36:28.184473+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:49.625268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":72888868,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":3425,"num_read_kb":11930,"num_write":200,"num_write_kb":2483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":1223716,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"97'205","reported_seq":5553,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.239304+0000","last_change":"2024-09-19T21:36:05.847174+0000","last_active":"2024-09-19T21:36:28.239304+0000","last_peered":"2024-09-19T21:36:28.239304+0000","last_clean":"2024-09-19T21:36:28.239304+0000","last_became_active":"2024-09-19T21:36:05.846322+0000","last_became_peered":"2024-09-19T21:36:05.846322+0000","last_unstale":"2024-09-19T21:36:28.239304+0000","last_undegraded":"2024-09-19T21:36:28.239304+0000","last_fullsized":"2024-09-19T21:36:28.239304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:34.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":73556480,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":5214,"num_read_kb":58823,"num_write":198,"num_write_kb":6613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"99'297","reported_seq":7423,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.200427+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:36:28.200427+0000","last_peered":"2024-09-19T21:36:28.200427+0000","last_clean":"2024-09-19T21:36:28.200427+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:36:28.200427+0000","last_undegraded":"2024-09-19T21:36:28.200427+0000","last_fullsized":"2024-09-19T21:36:28.200427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":34931370,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":7881,"num_read_kb":20465,"num_write":304,"num_write_kb":5391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1877140790,"num_objects":1864,"num_object_clones":528,"num_object_copies":3728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":719,"num_read":260375,"num_read_kb":878855,"num_write":13583,"num_write_kb":225851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1674,"num_bytes_recovered":109256338,"num_keys_recovered":253,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420421632,"data_stored":533314276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12782,"ondisk_log_size":12782,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574879,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94016,"kb_used_data":54700,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94277824,"statfs":{"total":96636764160,"available":96540491776,"internally_reserved":0,"allocated":56012800,"data_stored":63832928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88100000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0640000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.41499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.004,"5min":1.048,"15min":1.048},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.98399999999999999,"5min":0.996,"15min":0.996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.024,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.25,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.621},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2889999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.78900000000000003}]}]},{"osd":2,"up_from":0,"seq":55834574877,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142052,"kb_used_data":92884,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94229788,"statfs":{"total":96636764160,"available":96491302912,"internally_reserved":0,"allocated":95113216,"data_stored":126820352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.66700000000000004,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.75}]},{"osd":3,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.876},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.266,"5min":1.282,"15min":1.282},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.86599999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.90100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153112,"kb_used_data":95756,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94218728,"statfs":{"total":96636764160,"available":96479977472,"internally_reserved":0,"allocated":98054144,"data_stored":134338417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.96099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.87,"5min":1.333,"15min":1.333},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.012,"15min":1.012},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.052},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0900000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0740000000000001,"5min":1.718,"15min":1.718},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.90900000000000003}]}]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114656,"kb_used_data":83540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94257184,"statfs":{"total":96636764160,"available":96519356416,"internally_reserved":0,"allocated":85544960,"data_stored":99703179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92000000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.62}]},{"osd":1,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0269999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0489999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":1.175,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":3.2719999999999998,"5min":1.2330000000000001,"15min":0.89300000000000002},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":42.182000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.99399999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.752},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.105,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100536320,"data_stored":137473485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96325632,"data_stored":128697893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24182784,"data_stored":26125642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":84414464,"data_stored":99244086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":59662336,"data_stored":68143214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":55300096,"data_stored":73629956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:36:29.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:36:29.432 INFO:teuthology.orchestra.run.smithi122.stdout:5134 trunc from 0x33fe400 to 0x43b8200 2024-09-19T21:36:29.469 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.469 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.478 INFO:teuthology.orchestra.run.smithi122.stdout:5137 trunc from 0x43b8200 to 0xcd4d600 2024-09-19T21:36:29.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:29.495 INFO:teuthology.orchestra.run.smithi122.stdout:5138 read 0x843aa00 thru 0x843e742 (0x3d43 bytes) 2024-09-19T21:36:29.498 INFO:teuthology.orchestra.run.smithi122.stdout:5139 read 0x1bfce00 thru 0x1c0bb51 (0xed52 bytes) 2024-09-19T21:36:29.499 INFO:teuthology.orchestra.run.smithi122.stdout:5140 read 0x264d600 thru 0x265461d (0x701e bytes) 2024-09-19T21:36:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:5143 read 0x71de400 thru 0x71ed5fa (0xf1fb bytes) 2024-09-19T21:36:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:5145 read 0x78e6600 thru 0x78e68ad (0x2ae bytes) 2024-09-19T21:36:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:5146 write 0x4acb800 thru 0x4ad7380 (0xbb81 bytes) 2024-09-19T21:36:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:5147 read 0x61bfa00 thru 0x61cdbd6 (0xe1d7 bytes) 2024-09-19T21:36:29.501 INFO:teuthology.orchestra.run.smithi122.stdout:5150 punch from 0x4559400 to 0x455d600, (0x4200 bytes) 2024-09-19T21:36:29.501 INFO:teuthology.orchestra.run.smithi122.stdout:5152 punch from 0x6fd5200 to 0x6fdc200, (0x7000 bytes) 2024-09-19T21:36:29.501 INFO:teuthology.orchestra.run.smithi122.stdout:5154 write 0xaccac00 thru 0xacd9c7f (0xf080 bytes) 2024-09-19T21:36:29.502 INFO:teuthology.orchestra.run.smithi122.stdout:5156 write 0xae5d800 thru 0xae60037 (0x2838 bytes) 2024-09-19T21:36:29.502 INFO:teuthology.orchestra.run.smithi122.stdout:5157 write 0xa478800 thru 0xa4836d8 (0xaed9 bytes) 2024-09-19T21:36:29.502 INFO:teuthology.orchestra.run.smithi122.stdout:5159 write 0x234e600 thru 0x235c6b3 (0xe0b4 bytes) 2024-09-19T21:36:29.503 INFO:teuthology.orchestra.run.smithi122.stdout:5162 punch from 0x5b5ca00 to 0x5b64e00, (0x8400 bytes) 2024-09-19T21:36:29.505 INFO:teuthology.orchestra.run.smithi122.stdout:5163 write 0x9610200 thru 0x961a100 (0x9f01 bytes) 2024-09-19T21:36:29.505 INFO:teuthology.orchestra.run.smithi122.stdout:5167 punch from 0x5194800 to 0x51a0e00, (0xc600 bytes) 2024-09-19T21:36:29.506 INFO:teuthology.orchestra.run.smithi122.stdout:5170 read 0xbd4c00 thru 0xbe2f2d (0xe32e bytes) 2024-09-19T21:36:29.510 INFO:teuthology.orchestra.run.smithi122.stdout:5171 write 0xa4e8a00 thru 0xa4f13dc (0x89dd bytes) 2024-09-19T21:36:29.511 INFO:teuthology.orchestra.run.smithi122.stdout:5172 write 0xc9a9200 thru 0xc9b6d60 (0xdb61 bytes) 2024-09-19T21:36:29.511 INFO:teuthology.orchestra.run.smithi122.stdout:5175 write 0xa12d600 thru 0xa13b588 (0xdf89 bytes) 2024-09-19T21:36:29.511 INFO:teuthology.orchestra.run.smithi122.stdout:5176 read 0x188bc00 thru 0x189a0f5 (0xe4f6 bytes) 2024-09-19T21:36:29.519 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:29.515+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:29.556 INFO:teuthology.orchestra.run.smithi122.stdout:5180 write 0xec35600 thru 0xec44b8f (0xf590 bytes) 2024-09-19T21:36:29.558 INFO:teuthology.orchestra.run.smithi122.stdout:5181 punch from 0xe5dc00 to 0xe6a000, (0xc400 bytes) 2024-09-19T21:36:29.596 INFO:teuthology.orchestra.run.smithi122.stdout:5182 trunc from 0xec44c00 to 0x1142e00 2024-09-19T21:36:29.599 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.599 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:29.657 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:29.655+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:29.657 INFO:teuthology.orchestra.run.smithi122.stdout:5184 trunc from 0x1142e00 to 0x7c5ec00 2024-09-19T21:36:29.676 INFO:teuthology.orchestra.run.smithi122.stdout:5185 write 0x684aa00 thru 0x6851bf6 (0x71f7 bytes) 2024-09-19T21:36:29.677 INFO:teuthology.orchestra.run.smithi122.stdout:5186 read 0x149a000 thru 0x14a8252 (0xe253 bytes) 2024-09-19T21:36:29.680 INFO:teuthology.orchestra.run.smithi122.stdout:5188 trunc from 0x7c5ec00 to 0x68b8a00 2024-09-19T21:36:29.720 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:29.715+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:29.721 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.721 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:29.736 INFO:teuthology.orchestra.run.smithi122.stdout:5189 write 0x9f62c00 thru 0x9f65ef7 (0x32f8 bytes) 2024-09-19T21:36:29.737 INFO:teuthology.orchestra.run.smithi122.stdout:5191 read 0x6ebf200 thru 0x6ec0cc5 (0x1ac6 bytes) 2024-09-19T21:36:29.740 INFO:teuthology.orchestra.run.smithi122.stdout:5192 trunc from 0x9f66000 to 0x8929600 2024-09-19T21:36:29.800 INFO:teuthology.orchestra.run.smithi122.stdout:5194 write 0xe26a000 thru 0xe2749c7 (0xa9c8 bytes) 2024-09-19T21:36:29.802 INFO:teuthology.orchestra.run.smithi122.stdout:5195 read 0x488ec00 thru 0x488f617 (0xa18 bytes) 2024-09-19T21:36:29.802 INFO:teuthology.orchestra.run.smithi122.stdout:5196 punch from 0x176400 to 0x180000, (0x9c00 bytes) 2024-09-19T21:36:29.803 INFO:teuthology.orchestra.run.smithi122.stdout:5197 punch from 0x4163600 to 0x4168800, (0x5200 bytes) 2024-09-19T21:36:29.803 INFO:teuthology.orchestra.run.smithi122.stdout:5198 read 0x5902200 thru 0x59114c6 (0xf2c7 bytes) 2024-09-19T21:36:29.814 INFO:teuthology.orchestra.run.smithi122.stdout:5199 trunc from 0xe274a00 to 0x94df200 2024-09-19T21:36:29.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:29.821 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:36:29.821 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:29.819+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:29.834 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-19T21:36:29.001257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1877731177,"num_objects":1868,"num_object_clones":528,"num_object_copies":3736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1868,"num_whiteouts":719,"num_read":260491,"num_read_kb":878955,"num_write":13744,"num_write_kb":228269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1681,"num_bytes_recovered":110437112,"num_keys_recovered":253,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12902,"ondisk_log_size":12902,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":503836,"kb_used_data":326880,"kb_used_omap":6,"kb_used_meta":176889,"kb_avail":376983524,"statfs":{"total":386547056640,"available":386031128576,"internally_reserved":0,"allocated":334725120,"data_stored":424694876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":181135130},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":116444189,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":28,"num_read":5244,"num_read_kb":7616,"num_write":265,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.032525"},"pg_stats":[{"pgid":"3.1f","version":"94'319","reported_seq":3150,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.993021+0000","last_change":"2024-09-19T21:36:06.849962+0000","last_active":"2024-09-19T21:36:26.993021+0000","last_peered":"2024-09-19T21:36:26.993021+0000","last_clean":"2024-09-19T21:36:26.993021+0000","last_became_active":"2024-09-19T21:36:06.848734+0000","last_became_peered":"2024-09-19T21:36:06.848734+0000","last_unstale":"2024-09-19T21:36:26.993021+0000","last_undegraded":"2024-09-19T21:36:26.993021+0000","last_fullsized":"2024-09-19T21:36:26.993021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:12.838002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":44079749,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":17,"num_read":4356,"num_read_kb":35957,"num_write":322,"num_write_kb":22708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":24095839,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"97'207","reported_seq":1676,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.267548+0000","last_change":"2024-09-19T21:36:17.831424+0000","last_active":"2024-09-19T21:36:26.267548+0000","last_peered":"2024-09-19T21:36:26.267548+0000","last_clean":"2024-09-19T21:36:26.267548+0000","last_became_active":"2024-09-19T21:36:17.830193+0000","last_became_peered":"2024-09-19T21:36:17.830193+0000","last_unstale":"2024-09-19T21:36:26.267548+0000","last_undegraded":"2024-09-19T21:36:26.267548+0000","last_fullsized":"2024-09-19T21:36:26.267548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:32:51.816667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":33576004,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":1268,"num_read_kb":6951,"num_write":159,"num_write_kb":3211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1525316,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"99'1015","reported_seq":7216,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.144038+0000","last_change":"2024-09-19T21:36:18.837912+0000","last_active":"2024-09-19T21:36:28.144038+0000","last_peered":"2024-09-19T21:36:28.144038+0000","last_clean":"2024-09-19T21:36:28.144038+0000","last_became_active":"2024-09-19T21:36:18.836618+0000","last_became_peered":"2024-09-19T21:36:18.836618+0000","last_unstale":"2024-09-19T21:36:28.144038+0000","last_undegraded":"2024-09-19T21:36:28.144038+0000","last_fullsized":"2024-09-19T21:36:28.144038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:17:07.164147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":61966273,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":12637,"num_read_kb":31246,"num_write":1297,"num_write_kb":14357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":11559117,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"99'416","reported_seq":15555,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.084577+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:36:28.084577+0000","last_peered":"2024-09-19T21:36:28.084577+0000","last_clean":"2024-09-19T21:36:28.084577+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:36:28.084577+0000","last_undegraded":"2024-09-19T21:36:28.084577+0000","last_fullsized":"2024-09-19T21:36:28.084577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":43957300,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":16398,"num_read_kb":25708,"num_write":342,"num_write_kb":3883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"97'313","reported_seq":4017,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.443905+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:36:26.443905+0000","last_peered":"2024-09-19T21:36:26.443905+0000","last_clean":"2024-09-19T21:36:26.443905+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:36:26.443905+0000","last_undegraded":"2024-09-19T21:36:26.443905+0000","last_fullsized":"2024-09-19T21:36:26.443905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":177769161,"num_objects":65,"num_object_clones":18,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":3908,"num_read_kb":25123,"num_write":303,"num_write_kb":5636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"99'789","reported_seq":7303,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.262502+0000","last_change":"2024-09-19T21:36:05.849867+0000","last_active":"2024-09-19T21:36:28.262502+0000","last_peered":"2024-09-19T21:36:28.262502+0000","last_clean":"2024-09-19T21:36:28.262502+0000","last_became_active":"2024-09-19T21:36:05.848564+0000","last_became_peered":"2024-09-19T21:36:05.848564+0000","last_unstale":"2024-09-19T21:36:28.262502+0000","last_undegraded":"2024-09-19T21:36:28.262502+0000","last_fullsized":"2024-09-19T21:36:28.262502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.523212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":49090202,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":19,"num_read":10044,"num_read_kb":36348,"num_write":1221,"num_write_kb":8141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4294810,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"99'329","reported_seq":14020,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.247790+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:36:28.247790+0000","last_peered":"2024-09-19T21:36:28.247790+0000","last_clean":"2024-09-19T21:36:28.247790+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:36:28.247790+0000","last_undegraded":"2024-09-19T21:36:28.247790+0000","last_fullsized":"2024-09-19T21:36:28.247790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":39406786,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":20,"num_read":14484,"num_read_kb":30926,"num_write":413,"num_write_kb":8495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"97'228","reported_seq":8167,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.278339+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:36:26.278339+0000","last_peered":"2024-09-19T21:36:26.278339+0000","last_clean":"2024-09-19T21:36:26.278339+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:36:26.278339+0000","last_undegraded":"2024-09-19T21:36:26.278339+0000","last_fullsized":"2024-09-19T21:36:26.278339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":75472484,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":19,"num_read":8720,"num_read_kb":26159,"num_write":243,"num_write_kb":9085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"97'220","reported_seq":5651,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.241840+0000","last_change":"2024-09-19T21:36:05.848894+0000","last_active":"2024-09-19T21:36:28.241840+0000","last_peered":"2024-09-19T21:36:28.241840+0000","last_clean":"2024-09-19T21:36:28.241840+0000","last_became_active":"2024-09-19T21:36:05.848182+0000","last_became_peered":"2024-09-19T21:36:05.848182+0000","last_unstale":"2024-09-19T21:36:28.241840+0000","last_undegraded":"2024-09-19T21:36:28.241840+0000","last_fullsized":"2024-09-19T21:36:28.241840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:18.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":47604736,"num_objects":71,"num_object_clones":26,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":32,"num_read":5509,"num_read_kb":11213,"num_write":178,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"97'256","reported_seq":10358,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.782898+0000","last_change":"2024-09-19T21:36:17.832459+0000","last_active":"2024-09-19T21:36:26.782898+0000","last_peered":"2024-09-19T21:36:26.782898+0000","last_clean":"2024-09-19T21:36:26.782898+0000","last_became_active":"2024-09-19T21:36:17.831620+0000","last_became_peered":"2024-09-19T21:36:17.831620+0000","last_unstale":"2024-09-19T21:36:26.782898+0000","last_undegraded":"2024-09-19T21:36:26.782898+0000","last_fullsized":"2024-09-19T21:36:26.782898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:38:39.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":74689064,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":11271,"num_read_kb":28621,"num_write":210,"num_write_kb":5650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4500008,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"94'273","reported_seq":6135,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.485949+0000","last_change":"2024-09-19T21:36:05.846800+0000","last_active":"2024-09-19T21:36:27.485949+0000","last_peered":"2024-09-19T21:36:27.485949+0000","last_clean":"2024-09-19T21:36:27.485949+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:36:27.485949+0000","last_undegraded":"2024-09-19T21:36:27.485949+0000","last_fullsized":"2024-09-19T21:36:27.485949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":14478627,"num_objects":58,"num_object_clones":18,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":5653,"num_read_kb":10209,"num_write":223,"num_write_kb":2812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"98'250","reported_seq":9310,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.251000+0000","last_change":"2024-09-19T21:36:17.833402+0000","last_active":"2024-09-19T21:36:28.251000+0000","last_peered":"2024-09-19T21:36:28.251000+0000","last_clean":"2024-09-19T21:36:28.251000+0000","last_became_active":"2024-09-19T21:36:17.832672+0000","last_became_peered":"2024-09-19T21:36:17.832672+0000","last_unstale":"2024-09-19T21:36:28.251000+0000","last_undegraded":"2024-09-19T21:36:28.251000+0000","last_fullsized":"2024-09-19T21:36:28.251000+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:17:15.919793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":49711632,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":9412,"num_read_kb":25211,"num_write":211,"num_write_kb":4367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":3118608,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.993117+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:36:26.993117+0000","last_peered":"2024-09-19T21:36:26.993117+0000","last_clean":"2024-09-19T21:36:26.993117+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:36:26.993117+0000","last_undegraded":"2024-09-19T21:36:26.993117+0000","last_fullsized":"2024-09-19T21:36:26.993117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"96'395","reported_seq":8079,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989922+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:36:26.989922+0000","last_peered":"2024-09-19T21:36:26.989922+0000","last_clean":"2024-09-19T21:36:26.989922+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:36:26.989922+0000","last_undegraded":"2024-09-19T21:36:26.989922+0000","last_fullsized":"2024-09-19T21:36:26.989922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":70333570,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":8422,"num_read_kb":21078,"num_write":351,"num_write_kb":5762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989966+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:36:26.989966+0000","last_peered":"2024-09-19T21:36:26.989966+0000","last_clean":"2024-09-19T21:36:26.989966+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:36:26.989966+0000","last_undegraded":"2024-09-19T21:36:26.989966+0000","last_fullsized":"2024-09-19T21:36:26.989966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"97'570","reported_seq":5317,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.098936+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:36:28.098936+0000","last_peered":"2024-09-19T21:36:28.098936+0000","last_clean":"2024-09-19T21:36:28.098936+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:36:28.098936+0000","last_undegraded":"2024-09-19T21:36:28.098936+0000","last_fullsized":"2024-09-19T21:36:28.098936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":25442554,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":6646,"num_read_kb":13280,"num_write":633,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":95,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.002562+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:36:28.002562+0000","last_peered":"2024-09-19T21:36:28.002562+0000","last_clean":"2024-09-19T21:36:28.002562+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:36:28.002562+0000","last_undegraded":"2024-09-19T21:36:28.002562+0000","last_fullsized":"2024-09-19T21:36:28.002562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"98'296","reported_seq":10007,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.504328+0000","last_change":"2024-09-19T21:36:18.835965+0000","last_active":"2024-09-19T21:36:27.504328+0000","last_peered":"2024-09-19T21:36:27.504328+0000","last_clean":"2024-09-19T21:36:27.504328+0000","last_became_active":"2024-09-19T21:36:18.834188+0000","last_became_peered":"2024-09-19T21:36:18.834188+0000","last_unstale":"2024-09-19T21:36:27.504328+0000","last_undegraded":"2024-09-19T21:36:27.504328+0000","last_fullsized":"2024-09-19T21:36:27.504328+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:19:08.957546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":75759691,"num_objects":70,"num_object_clones":23,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":31,"num_read":9404,"num_read_kb":38414,"num_write":265,"num_write_kb":6829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":9899594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":99,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-19T21:36:24.972046+0000","last_change":"2024-09-19T21:36:14.898141+0000","last_active":"2024-09-19T21:36:24.972046+0000","last_peered":"2024-09-19T21:36:24.972046+0000","last_clean":"2024-09-19T21:36:24.972046+0000","last_became_active":"2024-09-19T21:36:14.897721+0000","last_became_peered":"2024-09-19T21:36:14.897721+0000","last_unstale":"2024-09-19T21:36:24.972046+0000","last_undegraded":"2024-09-19T21:36:24.972046+0000","last_fullsized":"2024-09-19T21:36:24.972046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:01:42.756456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"94'754","reported_seq":4872,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-19T21:36:24.971887+0000","last_change":"2024-09-19T21:36:17.832475+0000","last_active":"2024-09-19T21:36:24.971887+0000","last_peered":"2024-09-19T21:36:24.971887+0000","last_clean":"2024-09-19T21:36:24.971887+0000","last_became_active":"2024-09-19T21:36:17.831123+0000","last_became_peered":"2024-09-19T21:36:17.831123+0000","last_unstale":"2024-09-19T21:36:24.971887+0000","last_undegraded":"2024-09-19T21:36:24.971887+0000","last_fullsized":"2024-09-19T21:36:24.971887+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:17:29.475533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":40933084,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":7840,"num_read_kb":22219,"num_write":978,"num_write_kb":7884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":3776220,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"98'671","reported_seq":8299,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.610338+0000","last_change":"2024-09-19T21:36:17.835218+0000","last_active":"2024-09-19T21:36:27.610338+0000","last_peered":"2024-09-19T21:36:27.610338+0000","last_clean":"2024-09-19T21:36:27.610338+0000","last_became_active":"2024-09-19T21:36:17.834370+0000","last_became_peered":"2024-09-19T21:36:17.834370+0000","last_unstale":"2024-09-19T21:36:27.610338+0000","last_undegraded":"2024-09-19T21:36:27.610338+0000","last_fullsized":"2024-09-19T21:36:27.610338+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:56.289962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":59287845,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":17,"num_read":12301,"num_read_kb":17351,"num_write":730,"num_write_kb":3977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2192165,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":121,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006803+0000","last_change":"2024-09-19T21:36:02.839832+0000","last_active":"2024-09-19T21:36:28.006803+0000","last_peered":"2024-09-19T21:36:28.006803+0000","last_clean":"2024-09-19T21:36:28.006803+0000","last_became_active":"2024-09-19T21:36:02.830895+0000","last_became_peered":"2024-09-19T21:36:02.830895+0000","last_unstale":"2024-09-19T21:36:28.006803+0000","last_undegraded":"2024-09-19T21:36:28.006803+0000","last_fullsized":"2024-09-19T21:36:28.006803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:40.111547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"99'422","reported_seq":11062,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.205350+0000","last_change":"2024-09-19T21:36:17.834758+0000","last_active":"2024-09-19T21:36:28.205350+0000","last_peered":"2024-09-19T21:36:28.205350+0000","last_clean":"2024-09-19T21:36:28.205350+0000","last_became_active":"2024-09-19T21:36:17.832993+0000","last_became_peered":"2024-09-19T21:36:17.832993+0000","last_unstale":"2024-09-19T21:36:28.205350+0000","last_undegraded":"2024-09-19T21:36:28.205350+0000","last_fullsized":"2024-09-19T21:36:28.205350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:21:49.565848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":27430505,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":12256,"num_read_kb":42540,"num_write":378,"num_write_kb":8311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":8857810,"num_keys_recovered":59,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:25.984013+0000","last_change":"2024-09-19T21:36:02.831334+0000","last_active":"2024-09-19T21:36:25.984013+0000","last_peered":"2024-09-19T21:36:25.984013+0000","last_clean":"2024-09-19T21:36:25.984013+0000","last_became_active":"2024-09-19T21:36:02.830988+0000","last_became_peered":"2024-09-19T21:36:02.830988+0000","last_unstale":"2024-09-19T21:36:25.984013+0000","last_undegraded":"2024-09-19T21:36:25.984013+0000","last_fullsized":"2024-09-19T21:36:25.984013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:09:35.907712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"99'394","reported_seq":7223,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.246416+0000","last_change":"2024-09-19T21:36:17.836175+0000","last_active":"2024-09-19T21:36:28.246416+0000","last_peered":"2024-09-19T21:36:28.246416+0000","last_clean":"2024-09-19T21:36:28.246416+0000","last_became_active":"2024-09-19T21:36:17.835492+0000","last_became_peered":"2024-09-19T21:36:17.835492+0000","last_unstale":"2024-09-19T21:36:28.246416+0000","last_undegraded":"2024-09-19T21:36:28.246416+0000","last_fullsized":"2024-09-19T21:36:28.246416+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:56:59.854746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":83637797,"num_objects":68,"num_object_clones":24,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":27,"num_read":7643,"num_read_kb":30579,"num_write":488,"num_write_kb":9375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":5649957,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989496+0000","last_change":"2024-09-19T21:36:14.898393+0000","last_active":"2024-09-19T21:36:26.989496+0000","last_peered":"2024-09-19T21:36:26.989496+0000","last_clean":"2024-09-19T21:36:26.989496+0000","last_became_active":"2024-09-19T21:36:14.897906+0000","last_became_peered":"2024-09-19T21:36:14.897906+0000","last_unstale":"2024-09-19T21:36:26.989496+0000","last_undegraded":"2024-09-19T21:36:26.989496+0000","last_fullsized":"2024-09-19T21:36:26.989496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:39:57.143853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"97'350","reported_seq":10638,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.232391+0000","last_change":"2024-09-19T21:36:17.832488+0000","last_active":"2024-09-19T21:36:26.232391+0000","last_peered":"2024-09-19T21:36:26.232391+0000","last_clean":"2024-09-19T21:36:26.232391+0000","last_became_active":"2024-09-19T21:36:17.831369+0000","last_became_peered":"2024-09-19T21:36:17.831369+0000","last_unstale":"2024-09-19T21:36:26.232391+0000","last_undegraded":"2024-09-19T21:36:26.232391+0000","last_fullsized":"2024-09-19T21:36:26.232391+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:18:45.352469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":124156439,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":17,"num_read":11722,"num_read_kb":37195,"num_write":369,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":1162263,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":108,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.002183+0000","last_change":"2024-09-19T21:36:14.906859+0000","last_active":"2024-09-19T21:36:28.002183+0000","last_peered":"2024-09-19T21:36:28.002183+0000","last_clean":"2024-09-19T21:36:28.002183+0000","last_became_active":"2024-09-19T21:36:14.898975+0000","last_became_peered":"2024-09-19T21:36:14.898975+0000","last_unstale":"2024-09-19T21:36:28.002183+0000","last_undegraded":"2024-09-19T21:36:28.002183+0000","last_fullsized":"2024-09-19T21:36:28.002183+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:46:14.540137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":285,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.992571+0000","last_change":"2024-09-19T21:36:17.834100+0000","last_active":"2024-09-19T21:36:26.992571+0000","last_peered":"2024-09-19T21:36:26.992571+0000","last_clean":"2024-09-19T21:36:26.992571+0000","last_became_active":"2024-09-19T21:36:17.833564+0000","last_became_peered":"2024-09-19T21:36:17.833564+0000","last_unstale":"2024-09-19T21:36:26.992571+0000","last_undegraded":"2024-09-19T21:36:26.992571+0000","last_fullsized":"2024-09-19T21:36:26.992571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:10.720624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"99'391","reported_seq":14667,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.202207+0000","last_change":"2024-09-19T21:36:18.835818+0000","last_active":"2024-09-19T21:36:28.202207+0000","last_peered":"2024-09-19T21:36:28.202207+0000","last_clean":"2024-09-19T21:36:28.202207+0000","last_became_active":"2024-09-19T21:36:18.834188+0000","last_became_peered":"2024-09-19T21:36:18.834188+0000","last_unstale":"2024-09-19T21:36:28.202207+0000","last_undegraded":"2024-09-19T21:36:28.202207+0000","last_fullsized":"2024-09-19T21:36:28.202207+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:17.296547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":104853592,"num_objects":86,"num_object_clones":25,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":32,"num_read":13985,"num_read_kb":55282,"num_write":414,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":7192152,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"99'349","reported_seq":5153,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006192+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:36:28.006192+0000","last_peered":"2024-09-19T21:36:28.006192+0000","last_clean":"2024-09-19T21:36:28.006192+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:36:28.006192+0000","last_undegraded":"2024-09-19T21:36:28.006192+0000","last_fullsized":"2024-09-19T21:36:28.006192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":102157039,"num_objects":57,"num_object_clones":19,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":5570,"num_read_kb":16754,"num_write":322,"num_write_kb":6319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"99'462","reported_seq":7533,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.006623+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:36:28.006623+0000","last_peered":"2024-09-19T21:36:28.006623+0000","last_clean":"2024-09-19T21:36:28.006623+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:36:28.006623+0000","last_undegraded":"2024-09-19T21:36:28.006623+0000","last_fullsized":"2024-09-19T21:36:28.006623+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":35885876,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":27,"num_read":8961,"num_read_kb":42656,"num_write":553,"num_write_kb":12839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"97'190","reported_seq":2009,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.130183+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:36:26.130183+0000","last_peered":"2024-09-19T21:36:26.130183+0000","last_clean":"2024-09-19T21:36:26.130183+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:36:26.130183+0000","last_undegraded":"2024-09-19T21:36:26.130183+0000","last_fullsized":"2024-09-19T21:36:26.130183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":22110808,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":1860,"num_read_kb":19504,"num_write":182,"num_write_kb":6102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"99'172","reported_seq":2459,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.240335+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:36:28.240335+0000","last_peered":"2024-09-19T21:36:28.240335+0000","last_clean":"2024-09-19T21:36:28.240335+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:36:28.240335+0000","last_undegraded":"2024-09-19T21:36:28.240335+0000","last_fullsized":"2024-09-19T21:36:28.240335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":8029184,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":2022,"num_read_kb":6752,"num_write":134,"num_write_kb":2508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"99'332","reported_seq":8321,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.253273+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:36:28.253273+0000","last_peered":"2024-09-19T21:36:28.253273+0000","last_clean":"2024-09-19T21:36:28.253273+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:36:28.253273+0000","last_undegraded":"2024-09-19T21:36:28.253273+0000","last_fullsized":"2024-09-19T21:36:28.253273+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":115963994,"num_objects":77,"num_object_clones":27,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":31,"num_read":8248,"num_read_kb":19476,"num_write":351,"num_write_kb":9155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"94'489","reported_seq":5751,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:26.989537+0000","last_change":"2024-09-19T21:36:17.840819+0000","last_active":"2024-09-19T21:36:26.989537+0000","last_peered":"2024-09-19T21:36:26.989537+0000","last_clean":"2024-09-19T21:36:26.989537+0000","last_became_active":"2024-09-19T21:36:17.839929+0000","last_became_peered":"2024-09-19T21:36:17.839929+0000","last_unstale":"2024-09-19T21:36:26.989537+0000","last_undegraded":"2024-09-19T21:36:26.989537+0000","last_fullsized":"2024-09-19T21:36:26.989537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.623621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":33044532,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":7868,"num_read_kb":50770,"num_write":415,"num_write_kb":6304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":1322036,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"99'684","reported_seq":3984,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.227838+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:36:28.227838+0000","last_peered":"2024-09-19T21:36:28.227838+0000","last_clean":"2024-09-19T21:36:28.227838+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:36:28.227838+0000","last_undegraded":"2024-09-19T21:36:28.227838+0000","last_fullsized":"2024-09-19T21:36:28.227838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":10553711,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":6510,"num_read_kb":9984,"num_write":777,"num_write_kb":2774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"97'469","reported_seq":7224,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-19T21:36:27.483206+0000","last_change":"2024-09-19T21:36:05.849347+0000","last_active":"2024-09-19T21:36:27.483206+0000","last_peered":"2024-09-19T21:36:27.483206+0000","last_clean":"2024-09-19T21:36:27.483206+0000","last_became_active":"2024-09-19T21:36:05.848208+0000","last_became_peered":"2024-09-19T21:36:05.848208+0000","last_unstale":"2024-09-19T21:36:27.483206+0000","last_undegraded":"2024-09-19T21:36:27.483206+0000","last_fullsized":"2024-09-19T21:36:27.483206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:25:57.433432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":44381833,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":35,"num_read":8937,"num_read_kb":50131,"num_write":419,"num_write_kb":15540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":13819481,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"99'275","reported_seq":3156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.184473+0000","last_change":"2024-09-19T21:36:17.832573+0000","last_active":"2024-09-19T21:36:28.184473+0000","last_peered":"2024-09-19T21:36:28.184473+0000","last_clean":"2024-09-19T21:36:28.184473+0000","last_became_active":"2024-09-19T21:36:17.831693+0000","last_became_peered":"2024-09-19T21:36:17.831693+0000","last_unstale":"2024-09-19T21:36:28.184473+0000","last_undegraded":"2024-09-19T21:36:28.184473+0000","last_fullsized":"2024-09-19T21:36:28.184473+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:49.625268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":72888868,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":3425,"num_read_kb":11930,"num_write":200,"num_write_kb":2483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":1223716,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"97'205","reported_seq":5553,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.239304+0000","last_change":"2024-09-19T21:36:05.847174+0000","last_active":"2024-09-19T21:36:28.239304+0000","last_peered":"2024-09-19T21:36:28.239304+0000","last_clean":"2024-09-19T21:36:28.239304+0000","last_became_active":"2024-09-19T21:36:05.846322+0000","last_became_peered":"2024-09-19T21:36:05.846322+0000","last_unstale":"2024-09-19T21:36:28.239304+0000","last_undegraded":"2024-09-19T21:36:28.239304+0000","last_fullsized":"2024-09-19T21:36:28.239304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:34.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":73556480,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":5214,"num_read_kb":58823,"num_write":198,"num_write_kb":6613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"99'297","reported_seq":7423,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T21:36:28.200427+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:36:28.200427+0000","last_peered":"2024-09-19T21:36:28.200427+0000","last_clean":"2024-09-19T21:36:28.200427+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:36:28.200427+0000","last_undegraded":"2024-09-19T21:36:28.200427+0000","last_fullsized":"2024-09-19T21:36:28.200427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":34931370,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":7881,"num_read_kb":20465,"num_write":304,"num_write_kb":5391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1877140790,"num_objects":1864,"num_object_clones":528,"num_object_copies":3728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":719,"num_read":260375,"num_read_kb":878855,"num_write":13583,"num_write_kb":225851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1674,"num_bytes_recovered":109256338,"num_keys_recovered":253,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420421632,"data_stored":533314276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12782,"ondisk_log_size":12782,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574879,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94016,"kb_used_data":54700,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94277824,"statfs":{"total":96636764160,"available":96540491776,"internally_reserved":0,"allocated":56012800,"data_stored":63832928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88100000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0640000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.41499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.004,"5min":1.048,"15min":1.048},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.98399999999999999,"5min":0.996,"15min":0.996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.024,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.25,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.621},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2889999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.78900000000000003}]}]},{"osd":2,"up_from":0,"seq":55834574877,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142052,"kb_used_data":92884,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94229788,"statfs":{"total":96636764160,"available":96491302912,"internally_reserved":0,"allocated":95113216,"data_stored":126820352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.66700000000000004,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.75}]},{"osd":3,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.81299999999999994}]},{"osd":4,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.876},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.266,"5min":1.282,"15min":1.282},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.86599999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.90100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153112,"kb_used_data":95756,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94218728,"statfs":{"total":96636764160,"available":96479977472,"internally_reserved":0,"allocated":98054144,"data_stored":134338417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.96099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.87,"5min":1.333,"15min":1.333},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.012,"15min":1.012},"last":0.39600000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.052},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0900000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0740000000000001,"5min":1.718,"15min":1.718},"last":0.73799999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.90900000000000003}]}]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114656,"kb_used_data":83540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94257184,"statfs":{"total":96636764160,"available":96519356416,"internally_reserved":0,"allocated":85544960,"data_stored":99703179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92000000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.62}]},{"osd":1,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0269999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.0489999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":1.175,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":3.2719999999999998,"5min":1.2330000000000001,"15min":0.89300000000000002},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":42.182000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.99399999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.752},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.105,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100536320,"data_stored":137473485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96325632,"data_stored":128697893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24182784,"data_stored":26125642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":84414464,"data_stored":99244086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":59662336,"data_stored":68143214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":55300096,"data_stored":73629956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:36:29.836 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-19T21:36:29.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:29.837 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-19T21:36:29.837 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-19T21:36:29.837 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.838 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.841 INFO:teuthology.orchestra.run.smithi122.stdout:5200 punch from 0x5359200 to 0x5367a00, (0xe800 bytes) 2024-09-19T21:36:29.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:29.863 INFO:teuthology.orchestra.run.smithi122.stdout:5201 write 0xb0c5a00 thru 0xb0c9d33 (0x4334 bytes) 2024-09-19T21:36:29.866 INFO:teuthology.orchestra.run.smithi122.stdout:5202 write 0x7d82600 thru 0x7d8d154 (0xab55 bytes) 2024-09-19T21:36:29.866 INFO:teuthology.orchestra.run.smithi122.stdout:5203 write 0x5d57a00 thru 0x5d618d7 (0x9ed8 bytes) 2024-09-19T21:36:29.900 INFO:teuthology.orchestra.run.smithi122.stdout:5204 write 0xcc7e000 thru 0xcc80c85 (0x2c86 bytes) 2024-09-19T21:36:29.901 INFO:teuthology.orchestra.run.smithi122.stdout:5206 read 0xc78d400 thru 0xc79506b (0x7c6c bytes) 2024-09-19T21:36:29.902 INFO:teuthology.orchestra.run.smithi122.stdout:5207 write 0x4819c00 thru 0x4822686 (0x8a87 bytes) 2024-09-19T21:36:29.902 INFO:teuthology.orchestra.run.smithi122.stdout:5210 read 0x348f800 thru 0x3495edf (0x66e0 bytes) 2024-09-19T21:36:29.903 INFO:teuthology.orchestra.run.smithi122.stdout:5211 trunc from 0xcc80e00 to 0xc6f3e00 2024-09-19T21:36:29.922 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:29.919+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:29.945 INFO:teuthology.orchestra.run.smithi122.stdout:5213 write 0xac01c00 thru 0xac0cb5b (0xaf5c bytes) 2024-09-19T21:36:29.947 INFO:teuthology.orchestra.run.smithi122.stdout:5214 read 0x98ee000 thru 0x98f9111 (0xb112 bytes) 2024-09-19T21:36:29.947 INFO:teuthology.orchestra.run.smithi122.stdout:5215 read 0xbd12400 thru 0xbd1f86e (0xd46f bytes) 2024-09-19T21:36:29.947 INFO:teuthology.orchestra.run.smithi122.stdout:5217 read 0x96fbc00 thru 0x96fc5e9 (0x9ea bytes) 2024-09-19T21:36:29.948 INFO:teuthology.orchestra.run.smithi122.stdout:5218 write 0x3ff6200 thru 0x3ffb68e (0x548f bytes) 2024-09-19T21:36:29.948 INFO:teuthology.orchestra.run.smithi122.stdout:5222 read 0x6b41c00 thru 0x6b441fb (0x25fc bytes) 2024-09-19T21:36:29.948 INFO:teuthology.orchestra.run.smithi122.stdout:5223 read 0x4896600 thru 0x4899e1c (0x381d bytes) 2024-09-19T21:36:29.948 INFO:teuthology.orchestra.run.smithi122.stdout:5226 punch from 0xa164200 to 0xa16ca00, (0x8800 bytes) 2024-09-19T21:36:29.950 INFO:teuthology.orchestra.run.smithi122.stdout:5228 write 0x8c63000 thru 0x8c64e0b (0x1e0c bytes) 2024-09-19T21:36:29.950 INFO:teuthology.orchestra.run.smithi122.stdout:5230 write 0xc6c5e00 thru 0xc6d2d99 (0xcf9a bytes) 2024-09-19T21:36:29.950 INFO:teuthology.orchestra.run.smithi122.stdout:5231 write 0x831c000 thru 0x8327853 (0xb854 bytes) 2024-09-19T21:36:29.951 INFO:teuthology.orchestra.run.smithi122.stdout:5233 punch from 0x85c4e00 to 0x85cd000, (0x8200 bytes) 2024-09-19T21:36:29.952 INFO:teuthology.orchestra.run.smithi122.stdout:5235 read 0x55fca00 thru 0x56032da (0x68db bytes) 2024-09-19T21:36:29.952 INFO:teuthology.orchestra.run.smithi122.stdout:5236 write 0x186ba00 thru 0x1871f59 (0x655a bytes) 2024-09-19T21:36:29.970 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:29.970 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:29.981 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:29.988 INFO:teuthology.orchestra.run.smithi122.stdout:5238 write 0xedfaa00 thru 0xee06c20 (0xc221 bytes) 2024-09-19T21:36:29.989 INFO:teuthology.orchestra.run.smithi122.stdout:5239 punch from 0x94ce800 to 0x94d2200, (0x3a00 bytes) 2024-09-19T21:36:29.991 INFO:teuthology.orchestra.run.smithi122.stdout:5240 punch from 0x82c5800 to 0x82d1600, (0xbe00 bytes) 2024-09-19T21:36:29.991 INFO:teuthology.orchestra.run.smithi122.stdout:5242 read 0xe8b9a00 thru 0xe8bdd14 (0x4315 bytes) 2024-09-19T21:36:29.991 INFO:teuthology.orchestra.run.smithi122.stdout:5243 read 0x678ac00 thru 0x679476d (0x9b6e bytes) 2024-09-19T21:36:29.992 INFO:teuthology.orchestra.run.smithi122.stdout:5245 write 0x157400 thru 0x161a0e (0xa60f bytes) 2024-09-19T21:36:29.993 INFO:teuthology.orchestra.run.smithi122.stdout:5246 write 0x4664000 thru 0x4667d9f (0x3da0 bytes) 2024-09-19T21:36:29.993 INFO:teuthology.orchestra.run.smithi122.stdout:5250 write 0x9090000 thru 0x9099d98 (0x9d99 bytes) 2024-09-19T21:36:29.993 INFO:teuthology.orchestra.run.smithi122.stdout:5254 read 0xc69fc00 thru 0xc6a0c06 (0x1007 bytes) 2024-09-19T21:36:29.994 INFO:teuthology.orchestra.run.smithi122.stdout:5255 write 0x24f0a00 thru 0x24fe450 (0xda51 bytes) 2024-09-19T21:36:30.002 INFO:teuthology.orchestra.run.smithi122.stdout:5256 trunc from 0xee06e00 to 0xc260400 2024-09-19T21:36:30.023 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:36:30.023 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:30.019+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:30.036 INFO:teuthology.orchestra.run.smithi122.stdout:5257 read 0x9f8600 thru 0xa049ee (0xc3ef bytes) 2024-09-19T21:36:30.040 INFO:teuthology.orchestra.run.smithi122.stdout:5258 read 0x260ec00 thru 0x26169a5 (0x7da6 bytes) 2024-09-19T21:36:30.041 INFO:teuthology.orchestra.run.smithi122.stdout:5259 write 0xe36200 thru 0xe3d5a5 (0x73a6 bytes) 2024-09-19T21:36:30.042 INFO:teuthology.orchestra.run.smithi122.stdout:5260 write 0x8bf7800 thru 0x8bfe200 (0x6a01 bytes) 2024-09-19T21:36:30.042 INFO:teuthology.orchestra.run.smithi122.stdout:5265 write 0x9715000 thru 0x972154d (0xc54e bytes) 2024-09-19T21:36:30.042 INFO:teuthology.orchestra.run.smithi122.stdout:5266 read 0xec8c00 thru 0xed2572 (0x9973 bytes) 2024-09-19T21:36:30.052 INFO:teuthology.orchestra.run.smithi122.stdout:5267 trunc from 0xc260400 to 0x93b5a00 2024-09-19T21:36:30.091 INFO:teuthology.orchestra.run.smithi122.stdout:5268 read 0x82ea000 thru 0x82f58fb (0xb8fc bytes) 2024-09-19T21:36:30.093 INFO:teuthology.orchestra.run.smithi122.stdout:5269 read 0x327e00 thru 0x32ef0f (0x7110 bytes) 2024-09-19T21:36:30.095 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.095 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.095 INFO:teuthology.orchestra.run.smithi122.stdout:5271 read 0x8e5a00 thru 0x8e9357 (0x3958 bytes) 2024-09-19T21:36:30.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:30.109 INFO:teuthology.orchestra.run.smithi122.stdout:5274 trunc from 0x93b5a00 to 0xdc0b800 2024-09-19T21:36:30.123 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:30.119+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:30.126 INFO:teuthology.orchestra.run.smithi122.stdout:5281 read 0x73dca00 thru 0x73dd2ba (0x8bb bytes) 2024-09-19T21:36:30.129 INFO:teuthology.orchestra.run.smithi122.stdout:5282 write 0x1925800 thru 0x193071c (0xaf1d bytes) 2024-09-19T21:36:30.129 INFO:teuthology.orchestra.run.smithi122.stdout:5288 punch from 0x8340000 to 0x8343a00, (0x3a00 bytes) 2024-09-19T21:36:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:5289 read 0x133e600 thru 0x133ef57 (0x958 bytes) 2024-09-19T21:36:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:5290 write 0x5f89600 thru 0x5f96454 (0xce55 bytes) 2024-09-19T21:36:30.130 INFO:teuthology.orchestra.run.smithi122.stdout:5292 punch from 0x312e00 to 0x31fe00, (0xd000 bytes) 2024-09-19T21:36:30.132 INFO:teuthology.orchestra.run.smithi122.stdout:5296 punch from 0x76ede00 to 0x76f5800, (0x7a00 bytes) 2024-09-19T21:36:30.132 INFO:teuthology.orchestra.run.smithi122.stdout:5297 punch from 0x18ce00 to 0x190e00, (0x4000 bytes) 2024-09-19T21:36:30.132 INFO:teuthology.orchestra.run.smithi122.stdout:5300 trunc from 0xdc0b800 to 0xdb73000 2024-09-19T21:36:30.164 INFO:teuthology.orchestra.run.smithi122.stdout:5302 read 0x2212a00 thru 0x221756c (0x4b6d bytes) 2024-09-19T21:36:30.165 INFO:teuthology.orchestra.run.smithi122.stdout:5306 write 0x5dc6800 thru 0x5dd48be (0xe0bf bytes) 2024-09-19T21:36:30.166 INFO:teuthology.orchestra.run.smithi122.stdout:5307 write 0x82f2600 thru 0x82f6604 (0x4005 bytes) 2024-09-19T21:36:30.166 INFO:teuthology.orchestra.run.smithi122.stdout:5309 read 0x146da00 thru 0x147ad03 (0xd304 bytes) 2024-09-19T21:36:30.166 INFO:teuthology.orchestra.run.smithi122.stdout:5311 write 0x7afa400 thru 0x7b03799 (0x939a bytes) 2024-09-19T21:36:30.183 INFO:teuthology.orchestra.run.smithi122.stdout:5312 trunc from 0xdb73000 to 0x622a400 2024-09-19T21:36:30.192 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:36:30.187+0000 7eff847dd700 -1 mon.a@0(leader).osd e100 definitely_dead 0 2024-09-19T21:36:30.218 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.218 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.225 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:30.223+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:30.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:30.255 INFO:teuthology.orchestra.run.smithi122.stdout:5313 trunc from 0x622a400 to 0xc4b0c00 2024-09-19T21:36:30.274 INFO:teuthology.orchestra.run.smithi122.stdout:5315 punch from 0xa056600 to 0xa05be00, (0x5800 bytes) 2024-09-19T21:36:30.276 INFO:teuthology.orchestra.run.smithi122.stdout:5316 read 0xa992600 thru 0xa997949 (0x534a bytes) 2024-09-19T21:36:30.276 INFO:teuthology.orchestra.run.smithi122.stdout:5317 write 0x20ff000 thru 0x21012f5 (0x22f6 bytes) 2024-09-19T21:36:30.305 INFO:teuthology.orchestra.run.smithi122.stdout:5319 write 0xe8eea00 thru 0xe8f0cb6 (0x22b7 bytes) 2024-09-19T21:36:30.324 INFO:teuthology.orchestra.run.smithi122.stdout:5321 trunc from 0xe8f0e00 to 0x7e96e00 2024-09-19T21:36:30.326 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:30.323+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:30.368 INFO:teuthology.orchestra.run.smithi122.stdout:5322 read 0x704f600 thru 0x705c580 (0xcf81 bytes) 2024-09-19T21:36:30.388 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.388 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:5323 trunc from 0x7e96e00 to 0xb5ec00 2024-09-19T21:36:30.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:30.426 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:30.423+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:30.452 INFO:teuthology.orchestra.run.smithi122.stdout:5327 trunc from 0xb5ec00 to 0x47f8600 2024-09-19T21:36:30.476 INFO:teuthology.orchestra.run.smithi122.stdout:5328 read 0x3701200 thru 0x37055af (0x43b0 bytes) 2024-09-19T21:36:30.502 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.502 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.509 INFO:teuthology.orchestra.run.smithi122.stdout:5335 write 0x957f000 thru 0x957f625 (0x626 bytes) 2024-09-19T21:36:30.510 INFO:teuthology.orchestra.run.smithi122.stdout:5336 read 0x3996400 thru 0x39a2701 (0xc302 bytes) 2024-09-19T21:36:30.510 INFO:teuthology.orchestra.run.smithi122.stdout:5337 read 0x22bea00 thru 0x22cd8d8 (0xeed9 bytes) 2024-09-19T21:36:30.511 INFO:teuthology.orchestra.run.smithi122.stdout:5338 trunc from 0x957f800 to 0x90daa00 2024-09-19T21:36:30.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:30.527 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:30.523+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:30.545 INFO:teuthology.orchestra.run.smithi122.stdout:5339 write 0x4c1ac00 thru 0x4c22eee (0x82ef bytes) 2024-09-19T21:36:30.547 INFO:teuthology.orchestra.run.smithi122.stdout:5342 punch from 0x16fcc00 to 0x1703600, (0x6a00 bytes) 2024-09-19T21:36:30.548 INFO:teuthology.orchestra.run.smithi122.stdout:5344 read 0x3a71200 thru 0x3a7d865 (0xc666 bytes) 2024-09-19T21:36:30.564 INFO:teuthology.orchestra.run.smithi122.stdout:5347 trunc from 0x90daa00 to 0x157d200 2024-09-19T21:36:30.603 INFO:teuthology.orchestra.run.smithi122.stdout:5348 punch from 0x14e9000 to 0x14ea800, (0x1800 bytes) 2024-09-19T21:36:30.606 INFO:teuthology.orchestra.run.smithi122.stdout:5349 punch from 0x610a00 to 0x620800, (0xfe00 bytes) 2024-09-19T21:36:30.606 INFO:teuthology.orchestra.run.smithi122.stdout:5352 punch from 0x11c9e00 to 0x11d7200, (0xd400 bytes) 2024-09-19T21:36:30.607 INFO:teuthology.orchestra.run.smithi122.stdout:5353 punch from 0xe65a00 to 0xe6b000, (0x5600 bytes) 2024-09-19T21:36:30.608 INFO:teuthology.orchestra.run.smithi122.stdout:5354 trunc from 0x157d200 to 0xd73200 2024-09-19T21:36:30.624 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.624 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.628 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:30.627+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:30.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:30.648 INFO:teuthology.orchestra.run.smithi122.stdout:5357 read 0xa1e00 thru 0xa7e65 (0x6066 bytes) 2024-09-19T21:36:30.650 INFO:teuthology.orchestra.run.smithi122.stdout:5358 read 0xb1fc00 thru 0xb2b1d6 (0xb5d7 bytes) 2024-09-19T21:36:30.652 INFO:teuthology.orchestra.run.smithi122.stdout:5359 read 0xa57000 thru 0xa5880f (0x1810 bytes) 2024-09-19T21:36:30.653 INFO:teuthology.orchestra.run.smithi122.stdout:5361 read 0x7c5000 thru 0x7cec8d (0x9c8e bytes) 2024-09-19T21:36:30.686 INFO:teuthology.orchestra.run.smithi122.stdout:5362 write 0x4818c00 thru 0x4820da3 (0x81a4 bytes) 2024-09-19T21:36:30.688 INFO:teuthology.orchestra.run.smithi122.stdout:5366 write 0x29eae00 thru 0x29f7522 (0xc723 bytes) 2024-09-19T21:36:30.688 INFO:teuthology.orchestra.run.smithi122.stdout:5368 punch from 0x19cc400 to 0x19da200, (0xde00 bytes) 2024-09-19T21:36:30.689 INFO:teuthology.orchestra.run.smithi122.stdout:5369 read 0x13f4400 thru 0x1401134 (0xcd35 bytes) 2024-09-19T21:36:30.689 INFO:teuthology.orchestra.run.smithi122.stdout:5371 read 0x41f200 thru 0x427566 (0x8367 bytes) 2024-09-19T21:36:30.692 INFO:teuthology.orchestra.run.smithi122.stdout:5373 punch from 0x3205e00 to 0x3214200, (0xe400 bytes) 2024-09-19T21:36:30.692 INFO:teuthology.orchestra.run.smithi122.stdout:5374 write 0x2a62800 thru 0x2a6e9e4 (0xc1e5 bytes) 2024-09-19T21:36:30.692 INFO:teuthology.orchestra.run.smithi122.stdout:5377 punch from 0x2d98c00 to 0x2da7400, (0xe800 bytes) 2024-09-19T21:36:30.693 INFO:teuthology.orchestra.run.smithi122.stdout:5381 write 0x3c29400 thru 0x3c33c05 (0xa806 bytes) 2024-09-19T21:36:30.694 INFO:teuthology.orchestra.run.smithi122.stdout:5382 read 0xc39e00 thru 0xc44371 (0xa572 bytes) 2024-09-19T21:36:30.694 INFO:teuthology.orchestra.run.smithi122.stdout:5383 punch from 0x440a000 to 0x440e200, (0x4200 bytes) 2024-09-19T21:36:30.695 INFO:teuthology.orchestra.run.smithi122.stdout:5386 flatten 2024-09-19T21:36:30.729 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:30.723+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:30.751 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.751 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:30.762 INFO:teuthology.orchestra.run.smithi122.stdout:5389 read 0x3082200 thru 0x30835a8 (0x13a9 bytes) 2024-09-19T21:36:30.766 INFO:teuthology.orchestra.run.smithi122.stdout:5390 trunc from 0x4820e00 to 0x2b60400 2024-09-19T21:36:30.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:30.822 INFO:teuthology.orchestra.run.smithi122.stdout:5393 write 0x3764200 thru 0x376d4cf (0x92d0 bytes) 2024-09-19T21:36:30.823 INFO:teuthology.orchestra.run.smithi122.stdout:5399 write 0xc33000 thru 0xc42f70 (0xff71 bytes) 2024-09-19T21:36:30.826 INFO:teuthology.orchestra.run.smithi122.stdout:5402 trunc from 0x376d600 to 0x2b5f200 2024-09-19T21:36:30.830 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:30.827+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:30.867 INFO:teuthology.orchestra.run.smithi122.stdout:5404 punch from 0x1470a00 to 0x1474000, (0x3600 bytes) 2024-09-19T21:36:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:30.887 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:30.896 INFO:teuthology.orchestra.run.smithi122.stdout:5408 write 0x6619e00 thru 0x66295b1 (0xf7b2 bytes) 2024-09-19T21:36:30.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:30.899 INFO:teuthology.orchestra.run.smithi122.stdout:5409 punch from 0x3490e00 to 0x3491200, (0x400 bytes) 2024-09-19T21:36:30.900 INFO:teuthology.orchestra.run.smithi122.stdout:5410 punch from 0x50ea800 to 0x50eb400, (0xc00 bytes) 2024-09-19T21:36:30.901 INFO:teuthology.orchestra.run.smithi122.stdout:5411 write 0xb61e00 thru 0xb6bb74 (0x9d75 bytes) 2024-09-19T21:36:30.901 INFO:teuthology.orchestra.run.smithi122.stdout:5412 write 0x8a1600 thru 0x8a9237 (0x7c38 bytes) 2024-09-19T21:36:30.931 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:30.927+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:30.932 INFO:teuthology.orchestra.run.smithi122.stdout:5419 write 0xab6ba00 thru 0xab6d640 (0x1c41 bytes) 2024-09-19T21:36:30.934 INFO:teuthology.orchestra.run.smithi122.stdout:5420 write 0x330be00 thru 0x3311990 (0x5b91 bytes) 2024-09-19T21:36:30.934 INFO:teuthology.orchestra.run.smithi122.stdout:5423 write 0x7fca600 thru 0x7fd5778 (0xb179 bytes) 2024-09-19T21:36:30.956 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:30.966 INFO:teuthology.orchestra.run.smithi122.stdout:5424 write 0xad2ce00 thru 0xad38215 (0xb416 bytes) 2024-09-19T21:36:30.983 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:30.999 INFO:teuthology.orchestra.run.smithi122.stdout:5425 write 0xcd6ba00 thru 0xcd764a5 (0xaaa6 bytes) 2024-09-19T21:36:31.005 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:31.019 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.019 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:31.019 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:31.019 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:31.019 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.020 INFO:teuthology.orchestra.run.smithi122.stdout:5428 trunc from 0xcd76600 to 0x39e9000 2024-09-19T21:36:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.2. 2024-09-19T21:36:31.026 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:36:31.031 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:31.027+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:31.034 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:31.037 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-19T21:36:31.038 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:31.058 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:31.078 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:31.081 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:31.082 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2568 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3784704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.134 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.773787364, 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068439537999999994, 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2568 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357498+0000", 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357500+0000", 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-06 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357504+0000", 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9929999999999997e-06 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357508+0000", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.366e-06 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425577+0000", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068069122999999995 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425670+0000", 2024-09-19T21:36:31.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3562999999999998e-05 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425938+0000", 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026753999999999998 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2569 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3866624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.773775055999998, 2024-09-19T21:36:31.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068765307999999997, 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2569 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:31.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357511+0000", 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357510+0000", 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357512+0000", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.759e-06 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357516+0000", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6059999999999999e-06 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.357519+0000", 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4340000000000001e-06 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.425993+0000", 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068474165000000004 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426054+0000", 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0807000000000002e-05 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.426276+0000", 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022182199999999999 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2739 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 3907584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.741043816999998, 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068805729999999996, 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2739 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390242+0000", 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390241+0000", 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390245+0000", 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8199999999999998e-06 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390251+0000", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7830000000000004e-06 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.390255+0000", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6369999999999998e-06 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458604+0000", 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068348983000000002 2024-09-19T21:36:31.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.458671+0000", 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6470000000000006e-05 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.459048+0000", 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037697499999999999 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3781 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7430144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511286124000002, 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068780709999999995, 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3781 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.149 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620000+0000", 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.619999+0000", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620002+0000", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0350000000000002e-06 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620009+0000", 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7719999999999997e-06 2024-09-19T21:36:31.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620016+0000", 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1620000000000002e-06 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688439+0000", 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068422384000000003 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688509+0000", 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0987000000000003e-05 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688780+0000", 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027079200000000001 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511262688999999, 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:31.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:31.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:31.158 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511240960999999, 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:31.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:31.163 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511220229000003, 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:31.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511185793000003, 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:31.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:31.172 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.511142995999997, 2024-09-19T21:36:31.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:31.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:31.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505917486000001, 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.178 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.182 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505895187999997, 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:31.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505864332999998, 2024-09-19T21:36:31.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:31.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:31.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505821623999999, 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:31.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505797637000001, 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:31.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:31.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505766233000003, 2024-09-19T21:36:31.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.205 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505741067000002, 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:31.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505695989000003, 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505669245, 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:31.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505655953000002, 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:31.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.505639033999998, 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:31.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:31.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.227 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.227 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.227 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.227 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.227 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:31.127+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:31.228 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:31.270 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:31.267+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:31.335 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:31.331+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:31.382 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:31.382 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.392 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:31.436 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:31.431+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:31.498 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:31.499 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:31.537 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:31.535+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:31.637 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:31.635+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 135.728320438, 2024-09-19T21:36:31.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:31.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:31.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:31.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.671 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408753107000003, 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.672 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:31.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408741341000002, 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:31.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:31.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408728447999998, 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:31.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:31.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408716425000001, 2024-09-19T21:36:31.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:31.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408698780999998, 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:31.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:31.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408682984999999, 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:31.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408671134000002, 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.701 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.705 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408602289999997, 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:31.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.40857415, 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:31.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.408551021000001, 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:31.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:31.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.719 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.393774401000002, 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.386438085999998, 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:31.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:31.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.386417195999996, 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:31.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:31.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.379341656000001, 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:31.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.379321112, 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:31.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.198967041000003, 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:31.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:31.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.198956613999997, 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:31.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.198946184999997, 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:31.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.198936121999999, 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:31.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:31.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.762 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:31.765 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:31.759+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:31.865 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:31.859+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:31.879 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.85813879500000001, 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.85817608400000001, 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:31.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:31.884 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86758988199999998, 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.86773846099999996, 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.86747673300000006, 2024-09-19T21:36:31.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.86771936100000002, 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:31.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:31.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.85821122000000005, 2024-09-19T21:36:31.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.85854440799999998, 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:31.897 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:31.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 4 2024-09-19T21:36:31.899 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:31.900 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:31.966 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:31.963+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:32.059 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:32.059 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:32.060 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:32.060 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:32.067 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:32.063+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:32.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:32.168 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:32.163+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:32.175 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:32.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.443639504999993, 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:32.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:32.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:32.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.841684260999997, 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:32.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:32.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:32.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.841238279000002, 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.187 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:32.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.191 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.841124100000002, 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:32.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:32.196 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.841107667000003, 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:32.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.841092111000002, 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.201 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:32.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:32.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840978042000003, 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:32.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:32.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840961079000003, 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:32.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:32.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840945095000002, 2024-09-19T21:36:32.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:32.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:32.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840927919999999, 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:32.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840910721999997, 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.225 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:32.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840893891, 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.233 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840877370000001, 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840854766, 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:32.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840838378000001, 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:32.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:32.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 45.840729496999998, 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:32.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1616984619999999, 2024-09-19T21:36:32.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:32.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.161585313, 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:32.261 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.1523197999999999, 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:32.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:32.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.152247375, 2024-09-19T21:36:32.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:32.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:32.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:32.271 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:32.273 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:32.273 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:32.271+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:32.374 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:32.371+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:32.395 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:32.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:32.395 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:32.395 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:32.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:32.475 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:32.471+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:32.517 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:32.517 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:32.517 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:32.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:32.576 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:32.571+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380420446000002, 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:32.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380406906999994, 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380381166999996, 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:32.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380366953000006, 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:32.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.657 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380346036000006, 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:32.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:32.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380330659999998, 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:32.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380315732, 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:32.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:32.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:32.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380263111999994, 2024-09-19T21:36:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380213534000006, 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:32.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:32.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380196319000007, 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:32.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380177180999993, 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:32.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380162333000001, 2024-09-19T21:36:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:32.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.380139339999999, 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:32.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:32.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:32.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.379407048000004, 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.379356737999998, 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.379294426000001, 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:32.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:32.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.377963793000006, 2024-09-19T21:36:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.377828214999994, 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.721 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:32.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.725 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.377751455999999, 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.726 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:32.730 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 78.377692732, 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:32.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:32.735 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:32.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:32.737 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:32.731+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:32.837 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:32.831+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:32.896 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:32.896 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:32.896 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:32.896 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:32.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:32.938 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:32.935+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:33.009 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:33.009 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:33.009 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:33.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:33.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:36:33.069 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:33.067+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:33.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.133 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.931433120999998, 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.138 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.930853209999995, 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:33.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:33.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.930790563000002, 2024-09-19T21:36:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:33.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.930724243, 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.930020565000007, 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:33.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929947743, 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929934005000007, 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:33.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:33.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929916316000003, 2024-09-19T21:36:33.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:33.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:33.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929865930000005, 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:33.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:33.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929839040999994, 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:33.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:33.181 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929798465000005, 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.186 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929776851, 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.929757652000006, 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:33.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:33.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22505 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 566272~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.195 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.441040158, 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070135791000000003, 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22505 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688086+0000", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688085+0000", 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688091+0000", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0120000000000002e-06 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688104+0000", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3232e-05 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688119+0000", 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4548e-05 2024-09-19T21:36:33.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758034+0000", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069915036999999999 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758116+0000", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2525e-05 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758222+0000", 2024-09-19T21:36:33.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000105579 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22506 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 579584~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.440772881000001, 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070146347999999997, 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22506 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688353+0000", 2024-09-19T21:36:33.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688130+0000", 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997764 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688361+0000", 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023171300000000001 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688378+0000", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6804e-05 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688391+0000", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2799999999999999e-05 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758296+0000", 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069905388999999998 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758377+0000", 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0359999999999996e-05 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758500+0000", 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012305199999999999 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.204 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.440440049999999, 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:33.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:33.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:33.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22509 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 619520~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.439892905999997, 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070160101000000002, 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22509 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.209 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689233+0000", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688714+0000", 2024-09-19T21:36:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9994802 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689240+0000", 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00052599400000000001 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689254+0000", 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4042e-05 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689266+0000", 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2605e-05 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759170+0000", 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069903921999999993 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759253+0000", 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2588000000000003e-05 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759393+0000", 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000140572 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.439816567999998, 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:33.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:33.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.439484292000003, 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:33.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:33.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22513 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 659456~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.439181366, 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070232578000000004, 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:33.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22513 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:33.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689945+0000", 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689669+0000", 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9997239 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689950+0000", 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00028074900000000001 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689963+0000", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3224000000000001e-05 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689970+0000", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.798e-06 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759908+0000", 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.069938618999999994 2024-09-19T21:36:33.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759987+0000", 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8454000000000005e-05 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.760177+0000", 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00019081600000000001 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:33.227 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:33.228 INFO:teuthology.orchestra.run.smithi122.stdout:5429 read 0x943800 thru 0x952e65 (0xf666 bytes) 2024-09-19T21:36:33.229 INFO:teuthology.orchestra.run.smithi122.stdout:5430 write 0x588ca00 thru 0x5891dcc (0x53cd bytes) 2024-09-19T21:36:33.229 INFO:teuthology.orchestra.run.smithi122.stdout:5434 write 0x43fcc00 thru 0x4403109 (0x650a bytes) 2024-09-19T21:36:33.229 INFO:teuthology.orchestra.run.smithi122.stdout:5437 trunc from 0x5891e00 to 0x83a1400 2024-09-19T21:36:33.229 INFO:teuthology.orchestra.run.smithi122.stdout:5439 read 0x3035200 thru 0x3040849 (0xb64a bytes) 2024-09-19T21:36:33.229 INFO:teuthology.orchestra.run.smithi122.stdout:5440 clone 36 order 19 su 32768 sc 4 2024-09-19T21:36:33.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:33.230 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:33.223+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:33.240 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:33.267 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:33.288 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:33.310 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:33.328 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:33.323+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:33.334 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:33.345 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:33.345 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:33.345 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:33.345 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:33.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:33.370 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:33.372 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:33.373 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:33.429 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:33.423+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:33.467 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:33.467 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:33.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:33.531 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:33.527+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:33.585 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:33.585 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:33.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:33.632 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:33.627+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:33.727 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:33.727 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:33.732 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:33.727+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:33.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:33.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:33.831+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:33.847 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:33.848 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:33.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:33.935 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:33.931+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:33.969 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:33.969 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:33.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:34.036 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:34.031+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:34.060 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:36:34.109 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.109 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:34.137 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:34.135+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:34.233 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.233 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.237 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:34.231+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:34.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:34.338 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:34.335+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:34.360 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.360 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.371 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:34.439 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:34.435+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:34.477 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.477 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:34.539 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:34.535+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:34.594 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.594 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:34.640 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:34.635+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:34.741 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:34.735+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:34.766 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.766 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:34.842 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:34.835+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:34.894 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:34.894 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:34.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:34.942 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:34.939+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:35.015 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:35.015 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:35.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:35.043 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:35.039+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:35.138 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:35.138 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:35.144 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:35.139+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:35.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:35.245 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:35.239+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:35.257 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:35.257 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:35.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:35.345 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:35.343+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:35.367 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:35.367 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:35.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:35.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:35.446 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:35.439+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:35.539 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:35.539 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:35.540 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:35.547 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:35.543+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:35.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:35.564 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:35.585 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:35.611 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:35.634 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:35.648 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:35.643+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:35.656 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:35.659 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:35.662 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:35.662 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:35.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:35.750 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:35.747+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:35.785 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:35.785 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:35.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:35.850 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:35.847+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:35.901 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:35.901 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 100.148587742, 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:35.902 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:35.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521248346, 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:35.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:35.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.52119588, 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:35.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.917 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521165332999999, 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521142736000002, 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:35.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:35.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521120283999998, 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:35.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521105988000002, 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:35.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521089361000001, 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:35.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521070725000001, 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:35.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521044060000001, 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:35.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521033848999998, 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.521021363000003, 2024-09-19T21:36:35.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517146773, 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:35.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:35.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517129253999997, 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:35.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517091700000002, 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:35.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:35.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517073857, 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517061408000004, 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:35.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517050716, 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.517039138999998, 2024-09-19T21:36:35.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:35.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 55.101067182999998, 2024-09-19T21:36:35.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:35.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:35.998 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:35.999 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:35.995+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:36.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:36.038 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:36.038 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:36.039 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T21:36:36.039 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:36:36.039 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:36:36.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T21:36:36.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:36:36.101 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:36.099+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:36.146 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.146 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:36.146 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:36.146 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:36.181 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.181 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "" 2024-09-19T21:36:36.181 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.201 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:36.199+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:36.271 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:36.272 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.284 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:36.302 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:36.299+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:36.403 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:36.399+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:36.412 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.790879232999998, 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.413 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:36.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:36.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.790857504999998, 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:36.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:36.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.790836773000002, 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.425 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:36.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.429 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.790802337000002, 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.790759540000003, 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:36.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:36.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785534030000001, 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.439 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:36.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.443 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785511732000003, 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:36.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785480876999998, 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.448 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.452 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785438167999999, 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:36.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:36.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785414181, 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:36.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:36.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785382777000002, 2024-09-19T21:36:36.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:36.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785357611000002, 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:36.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:36.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785312533000003, 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:36.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785285789, 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785272497000001, 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 63.785255577999997, 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.385268806, 2024-09-19T21:36:36.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:36:36.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:36:36.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.3837021030000001, 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.492 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:36.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:36:36.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:36:36.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.3836658230000003, 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:36:36.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:36:36.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:36:36.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:36:36.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.3836458010000001, 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:36:36.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:36:36.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:36:36.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.510 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.511 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:36.512 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:36.507+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:36.612 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:36.607+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:36.622 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.622 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:36.623 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:36.623 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:36.713 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:36.711+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:36.743 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:36.743 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:36.743 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:36.775 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-19T21:36:36.814 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:36.811+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:36.866 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 140.93110464099999, 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.867 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:36.868 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.869 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.870 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:36.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:36.872 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.873 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611537310000003, 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.875 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:36.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.877 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611525544000003, 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.879 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:36.880 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.881 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:36.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611512650999998, 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:36.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.884 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.885 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.886 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.887 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611500628000002, 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:36.888 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.889 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.890 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:36.891 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611482984000006, 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.892 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.894 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.895 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.896 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611467188000006, 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.897 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.898 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:36.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.900 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611455336999995, 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:36.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:36.905 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611386492999998, 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:36.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:36.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:36.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.909 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611358353, 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:36.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:36.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.914 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.611335224000001, 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:36.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.596558603999995, 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:36.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:36.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.589222289000006, 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:36.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.589201399000004, 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:36.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.582125859000001, 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:36.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:36.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.582105315000007, 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:36.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:36.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:36.942 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.401751243999996, 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:36.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:36.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:36.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.401740817000004, 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.947 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.951 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.401730388000004, 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:36.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.401720324999999, 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:36.956 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:36.960 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:36.962 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:36.962 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:36.955+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:37.061 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:36:37.062 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:37.059+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:118 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.066633+0000", 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.063301224000000003, 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063349064999999996, 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 118 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066633+0000", 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066633+0000", 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066631+0000", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066641+0000", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0438000000000001e-05 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066658+0000", 2024-09-19T21:36:37.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7076e-05 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066669+0000", 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0818999999999999e-05 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066787+0000", 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000118286 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066797+0000", 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9369999999999999e-06 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:124 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.066697+0000", 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.063237194999999996, 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063380847000000004, 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 124 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066697+0000", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066697+0000", 2024-09-19T21:36:37.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066696+0000", 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066701+0000", 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.515e-06 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066709+0000", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5059999999999995e-06 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066716+0000", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6630000000000004e-06 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066823+0000", 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000107157 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066830+0000", 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1760000000000004e-06 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:132 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.066847+0000", 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.063087004000000002, 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063310546999999995, 2024-09-19T21:36:37.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 132 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066847+0000", 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066847+0000", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066846+0000", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066852+0000", 2024-09-19T21:36:37.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0530000000000003e-06 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066862+0000", 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2439999999999999e-06 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.066869+0000", 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0639999999999996e-06 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070004+0000", 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031356589999999998 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070014+0000", 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0192e-05 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:138 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.069975+0000", 2024-09-19T21:36:37.143 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.059958887000000002, 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.060259949, 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 138 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.069975+0000", 2024-09-19T21:36:37.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.069975+0000", 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.069973+0000", 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.069982+0000", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2989999999999997e-06 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.069999+0000", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6911000000000001e-05 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070010+0000", 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1559000000000001e-05 2024-09-19T21:36:37.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070044+0000", 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.349e-05 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070051+0000", 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4109999999999999e-06 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:142 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.070021+0000", 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.059913122999999999, 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.060294970000000003, 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 142 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070021+0000", 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070021+0000", 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070021+0000", 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070025+0000", 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0999999999999997e-06 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070032+0000", 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6020000000000002e-06 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070039+0000", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2589999999999997e-06 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070092+0000", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3567999999999999e-05 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070099+0000", 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4930000000000003e-06 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.151 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:151 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.070044+0000", 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.059890132999999998, 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.060348565, 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 151 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070044+0000", 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070044+0000", 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070043+0000", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070047+0000", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1709999999999998e-06 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070053+0000", 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2160000000000003e-06 2024-09-19T21:36:37.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070066+0000", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3062e-05 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070117+0000", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1283999999999999e-05 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070122+0000", 2024-09-19T21:36:37.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2379999999999997e-06 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.6063.0:161 3.0 3.12e882c0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:37.070104+0000", 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.059830403999999997, 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.060365100999999997, 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:36:37.156 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.6063", 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 161 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070104+0000", 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.157 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070104+0000", 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070103+0000", 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070109+0000", 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.674e-06 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070120+0000", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1144e-05 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070128+0000", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.153e-06 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070164+0000", 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5992999999999999e-05 2024-09-19T21:36:37.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:37.070170+0000", 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6340000000000003e-06 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 7 2024-09-19T21:36:37.160 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:37.161 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:37.162 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:37.159+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:37.264 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:37.259+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:37.275 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:37.275 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:37.275 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:37.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:37.364 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:37.359+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:37.396 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:37.396 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:37.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:37.396 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.665622306, 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.397 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:37.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.063667062, 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.404 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:37.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:37.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.408 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.063221079999998, 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:37.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.063106900999998, 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:37.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:37.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.417 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.063090467999999, 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:37.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:37.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.063074911999998, 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:37.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062960842999999, 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:37.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062943879999999, 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:37.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062927895999998, 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.437 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.441 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062910721000002, 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:37.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:37.446 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062893523, 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.447 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:37.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062876692000003, 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062860170999997, 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:37.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062837567000003, 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:37.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:37.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062821178999997, 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:37.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.470 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.062712298000001, 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:37.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:37.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.475 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.3836812629999997, 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:37.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:37.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:37.480 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.383568114, 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:37.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:37.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:37.485 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.3743026010000001, 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:37.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.3742301760000002, 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.491 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:37.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:37.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:37.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:37.496 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:37.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:37.498 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:37.495+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:37.599 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:37.599+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:37.616 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:37.617 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:37.617 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:37.617 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:37.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:37.671 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:37.718 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:37.715+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:37.733 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:37.734 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:37.734 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:37.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:37.801 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:37.799+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:37.822 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:37.841 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:37.858 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:37.858 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:37.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:37.858 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:37.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600351833999994, 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600338295, 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:37.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.867 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600312555000002, 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:37.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600298340999998, 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.872 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600277423999998, 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:37.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600262048000005, 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600247120000006, 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:37.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600194500000001, 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600144921999998, 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600127706999999, 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:37.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:37.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:37.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.905 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600108569, 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:37.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600093720999993, 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:37.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:37.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.600070728000006, 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:37.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.919 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.599338435999996, 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.599288126000005, 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.599225813999993, 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:37.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.597895180999998, 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:37.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.597759603, 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:37.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:37.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.597682844000005, 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:37.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.948 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.597624120000006, 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:37.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:37.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:37.953 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:37.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:37.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:37.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:37.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:37.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:37.957 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:37.951+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:37.957 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:37.958 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:38.056 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:38.051+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:38.129 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.129 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.140 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:38.185 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:38.183+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:38.245 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.245 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:38.257 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:38.251+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:38.358 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:38.355+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:38.359 INFO:teuthology.orchestra.run.smithi122.stdout:5441 trunc from 0x83a1400 to 0xeb9c200 2024-09-19T21:36:38.364 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.364 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:38.388 INFO:teuthology.orchestra.run.smithi122.stdout:5442 write 0x556cc00 thru 0x556f494 (0x2895 bytes) 2024-09-19T21:36:38.430 INFO:teuthology.orchestra.run.smithi122.stdout:5444 trunc from 0xeb9c200 to 0x4615c00 2024-09-19T21:36:38.459 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:38.455+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:38.477 INFO:teuthology.orchestra.run.smithi122.stdout:5445 read 0x3aba00 thru 0x3ada4b (0x204c bytes) 2024-09-19T21:36:38.479 INFO:teuthology.orchestra.run.smithi122.stdout:5446 write 0x1b23000 thru 0x1b24459 (0x145a bytes) 2024-09-19T21:36:38.480 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.481 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:38.493 INFO:teuthology.orchestra.run.smithi122.stdout:5449 trunc from 0x4615c00 to 0xb20c800 2024-09-19T21:36:38.547 INFO:teuthology.orchestra.run.smithi122.stdout:5451 write 0xd3f7800 thru 0xd3f917a (0x197b bytes) 2024-09-19T21:36:38.548 INFO:teuthology.orchestra.run.smithi122.stdout:5452 write 0xc187c00 thru 0xc18de9d (0x629e bytes) 2024-09-19T21:36:38.549 INFO:teuthology.orchestra.run.smithi122.stdout:5454 write 0x55a6000 thru 0x55af141 (0x9142 bytes) 2024-09-19T21:36:38.549 INFO:teuthology.orchestra.run.smithi122.stdout:5456 punch from 0x6abcc00 to 0x6ac1a00, (0x4e00 bytes) 2024-09-19T21:36:38.559 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:38.555+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:38.595 INFO:teuthology.orchestra.run.smithi122.stdout:5458 trunc from 0xd3f9200 to 0xfd1400 2024-09-19T21:36:38.601 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.601 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:38.634 INFO:teuthology.orchestra.run.smithi122.stdout:5461 read 0x314600 thru 0x31731e (0x2d1f bytes) 2024-09-19T21:36:38.637 INFO:teuthology.orchestra.run.smithi122.stdout:5466 read 0x8ad800 thru 0x8b2817 (0x5018 bytes) 2024-09-19T21:36:38.648 INFO:teuthology.orchestra.run.smithi122.stdout:5467 trunc from 0xfd1400 to 0x534aa00 2024-09-19T21:36:38.660 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:38.655+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:38.696 INFO:teuthology.orchestra.run.smithi122.stdout:5468 write 0x82b4c00 thru 0x82b704b (0x244c bytes) 2024-09-19T21:36:38.699 INFO:teuthology.orchestra.run.smithi122.stdout:5469 write 0x5e9a600 thru 0x5ea5196 (0xab97 bytes) 2024-09-19T21:36:38.699 INFO:teuthology.orchestra.run.smithi122.stdout:5483 write 0x239f000 thru 0x23a600a (0x700b bytes) 2024-09-19T21:36:38.699 INFO:teuthology.orchestra.run.smithi122.stdout:5484 read 0x1db8800 thru 0x1db9c85 (0x1486 bytes) 2024-09-19T21:36:38.700 INFO:teuthology.orchestra.run.smithi122.stdout:5485 write 0x2a21000 thru 0x2a2cee4 (0xbee5 bytes) 2024-09-19T21:36:38.700 INFO:teuthology.orchestra.run.smithi122.stdout:5487 read 0x29b0000 thru 0x29b0ce4 (0xce5 bytes) 2024-09-19T21:36:38.700 INFO:teuthology.orchestra.run.smithi122.stdout:5488 punch from 0x37d6e00 to 0x37df200, (0x8400 bytes) 2024-09-19T21:36:38.702 INFO:teuthology.orchestra.run.smithi122.stdout:5489 read 0x53c3a00 thru 0x53cb0eb (0x76ec bytes) 2024-09-19T21:36:38.735 INFO:teuthology.orchestra.run.smithi122.stdout:5490 write 0xda81400 thru 0xda82908 (0x1509 bytes) 2024-09-19T21:36:38.736 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.737 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.738 INFO:teuthology.orchestra.run.smithi122.stdout:5494 trunc from 0xda82a00 to 0xcacdc00 2024-09-19T21:36:38.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:38.774 INFO:teuthology.orchestra.run.smithi122.stdout:5497 write 0x5be1e00 thru 0x5beeae8 (0xcce9 bytes) 2024-09-19T21:36:38.776 INFO:teuthology.orchestra.run.smithi122.stdout:5499 read 0xf0ba00 thru 0xf11d45 (0x6346 bytes) 2024-09-19T21:36:38.777 INFO:teuthology.orchestra.run.smithi122.stdout:5500 read 0x9e8000 thru 0x9f4eb1 (0xceb2 bytes) 2024-09-19T21:36:38.778 INFO:teuthology.orchestra.run.smithi122.stdout:5501 write 0x61acc00 thru 0x61ba313 (0xd714 bytes) 2024-09-19T21:36:38.778 INFO:teuthology.orchestra.run.smithi122.stdout:5502 read 0xbae2000 thru 0xbaebc9a (0x9c9b bytes) 2024-09-19T21:36:38.778 INFO:teuthology.orchestra.run.smithi122.stdout:5503 punch from 0xc1ca800 to 0xc1d4c00, (0xa400 bytes) 2024-09-19T21:36:38.780 INFO:teuthology.orchestra.run.smithi122.stdout:5506 read 0xa498400 thru 0xa49b874 (0x3475 bytes) 2024-09-19T21:36:38.793 INFO:teuthology.orchestra.run.smithi122.stdout:5507 trunc from 0xcacdc00 to 0x734f200 2024-09-19T21:36:38.798 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:38.795+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:38.821 INFO:teuthology.orchestra.run.smithi122.stdout:5508 read 0x6bf2e00 thru 0x6bfd83d (0xaa3e bytes) 2024-09-19T21:36:38.822 INFO:teuthology.orchestra.run.smithi122.stdout:5509 punch from 0x3e5bc00 to 0x3e68600, (0xca00 bytes) 2024-09-19T21:36:38.835 INFO:teuthology.orchestra.run.smithi122.stdout:5510 trunc from 0x734f200 to 0x11d1400 2024-09-19T21:36:38.861 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:38.859+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:38.868 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.868 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.873 INFO:teuthology.orchestra.run.smithi122.stdout:5511 trunc from 0x11d1400 to 0x2c76400 2024-09-19T21:36:38.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:38.931 INFO:teuthology.orchestra.run.smithi122.stdout:5513 write 0x9a7da00 thru 0x9a89d27 (0xc328 bytes) 2024-09-19T21:36:38.932 INFO:teuthology.orchestra.run.smithi122.stdout:5515 read 0x335ee00 thru 0x3365d48 (0x6f49 bytes) 2024-09-19T21:36:38.933 INFO:teuthology.orchestra.run.smithi122.stdout:5516 write 0x1fd9400 thru 0x1fde09f (0x4ca0 bytes) 2024-09-19T21:36:38.933 INFO:teuthology.orchestra.run.smithi122.stdout:5517 read 0x23b3a00 thru 0x23b4534 (0xb35 bytes) 2024-09-19T21:36:38.944 INFO:teuthology.orchestra.run.smithi122.stdout:5518 trunc from 0x9a89e00 to 0x51da200 2024-09-19T21:36:38.962 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:38.959+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:38.987 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:38.987 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:38.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:39.009 INFO:teuthology.orchestra.run.smithi122.stdout:5519 write 0x5a36400 thru 0x5a3b6ef (0x52f0 bytes) 2024-09-19T21:36:39.012 INFO:teuthology.orchestra.run.smithi122.stdout:5523 read 0x3b32800 thru 0x3b36376 (0x3b77 bytes) 2024-09-19T21:36:39.012 INFO:teuthology.orchestra.run.smithi122.stdout:5524 read 0x3af3000 thru 0x3affee4 (0xcee5 bytes) 2024-09-19T21:36:39.012 INFO:teuthology.orchestra.run.smithi122.stdout:5525 read 0x4c4fe00 thru 0x4c58e1a (0x901b bytes) 2024-09-19T21:36:39.062 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:39.059+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:39.063 INFO:teuthology.orchestra.run.smithi122.stdout:5528 write 0xd6bb000 thru 0xd6c40ee (0x90ef bytes) 2024-09-19T21:36:39.066 INFO:teuthology.orchestra.run.smithi122.stdout:5530 read 0xc5d6c00 thru 0xc5e12ab (0xa6ac bytes) 2024-09-19T21:36:39.066 INFO:teuthology.orchestra.run.smithi122.stdout:5531 write 0x470ca00 thru 0x4710354 (0x3955 bytes) 2024-09-19T21:36:39.066 INFO:teuthology.orchestra.run.smithi122.stdout:5533 write 0x6941800 thru 0x6941b77 (0x378 bytes) 2024-09-19T21:36:39.074 INFO:teuthology.orchestra.run.smithi122.stdout:5534 trunc from 0xd6c4200 to 0x9e63000 2024-09-19T21:36:39.078 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:36:39.115 INFO:teuthology.orchestra.run.smithi122.stdout:5535 write 0x803d600 thru 0x804caee (0xf4ef bytes) 2024-09-19T21:36:39.129 INFO:teuthology.orchestra.run.smithi122.stdout:5537 trunc from 0x9e63000 to 0x500d200 2024-09-19T21:36:39.131 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.132 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:39.163 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:39.159+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:39.169 INFO:teuthology.orchestra.run.smithi122.stdout:5541 write 0x41ffe00 thru 0x4207b7b (0x7d7c bytes) 2024-09-19T21:36:39.171 INFO:teuthology.orchestra.run.smithi122.stdout:5543 punch from 0x4ec1600 to 0x4ec6600, (0x5000 bytes) 2024-09-19T21:36:39.172 INFO:teuthology.orchestra.run.smithi122.stdout:5544 write 0x2bca200 thru 0x2bcb615 (0x1416 bytes) 2024-09-19T21:36:39.172 INFO:teuthology.orchestra.run.smithi122.stdout:5545 read 0x46e9400 thru 0x46f2d80 (0x9981 bytes) 2024-09-19T21:36:39.173 INFO:teuthology.orchestra.run.smithi122.stdout:5547 punch from 0x497a600 to 0x4984a00, (0xa400 bytes) 2024-09-19T21:36:39.174 INFO:teuthology.orchestra.run.smithi122.stdout:5550 read 0x915200 thru 0x917062 (0x1e63 bytes) 2024-09-19T21:36:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:5553 write 0xab8ba00 thru 0xab9443e (0x8a3f bytes) 2024-09-19T21:36:39.209 INFO:teuthology.orchestra.run.smithi122.stdout:5554 write 0x3d93a00 thru 0x3d9a905 (0x6f06 bytes) 2024-09-19T21:36:39.211 INFO:teuthology.orchestra.run.smithi122.stdout:5555 trunc from 0xab94600 to 0xb5db000 2024-09-19T21:36:39.237 INFO:teuthology.orchestra.run.smithi122.stdout:5559 read 0x9d74800 thru 0x9d759e0 (0x11e1 bytes) 2024-09-19T21:36:39.242 INFO:teuthology.orchestra.run.smithi122.stdout:5560 read 0x10e1800 thru 0x10e1ade (0x2df bytes) 2024-09-19T21:36:39.242 INFO:teuthology.orchestra.run.smithi122.stdout:5563 read 0x98b1c00 thru 0x98bf90a (0xdd0b bytes) 2024-09-19T21:36:39.261 INFO:teuthology.orchestra.run.smithi122.stdout:5564 trunc from 0xb5db000 to 0x13c1a00 2024-09-19T21:36:39.265 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:39.259+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:39.271 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.271 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.285 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:39.295 INFO:teuthology.orchestra.run.smithi122.stdout:5565 read 0xe87600 thru 0xe93b34 (0xc535 bytes) 2024-09-19T21:36:39.298 INFO:teuthology.orchestra.run.smithi122.stdout:5566 punch from 0xac4600 to 0xac6800, (0x2200 bytes) 2024-09-19T21:36:39.311 INFO:teuthology.orchestra.run.smithi122.stdout:5567 trunc from 0x13c1a00 to 0x6ce3200 2024-09-19T21:36:39.332 INFO:teuthology.orchestra.run.smithi122.stdout:5569 read 0x70f600 thru 0x71e50c (0xef0d bytes) 2024-09-19T21:36:39.345 INFO:teuthology.orchestra.run.smithi122.stdout:5572 trunc from 0x6ce3200 to 0x1dfc200 2024-09-19T21:36:39.365 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:39.363+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:39.369 INFO:teuthology.orchestra.run.smithi122.stdout:5573 flatten 2024-09-19T21:36:39.401 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.401 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:39.419 INFO:teuthology.orchestra.run.smithi122.stdout:5574 trunc from 0x1dfc200 to 0x122fc00 2024-09-19T21:36:39.440 INFO:teuthology.orchestra.run.smithi122.stdout:5575 read 0xfa2600 thru 0xfae391 (0xbd92 bytes) 2024-09-19T21:36:39.466 INFO:teuthology.orchestra.run.smithi122.stdout:5576 trunc from 0x122fc00 to 0xcef5e00 2024-09-19T21:36:39.466 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:39.463+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:39.486 INFO:teuthology.orchestra.run.smithi122.stdout:5578 read 0x3939200 thru 0x393ceb4 (0x3cb5 bytes) 2024-09-19T21:36:39.488 INFO:teuthology.orchestra.run.smithi122.stdout:5583 write 0x89e0400 thru 0x89e124c (0xe4d bytes) 2024-09-19T21:36:39.488 INFO:teuthology.orchestra.run.smithi122.stdout:5584 read 0xb883800 thru 0xb892f62 (0xf763 bytes) 2024-09-19T21:36:39.488 INFO:teuthology.orchestra.run.smithi122.stdout:5585 write 0x96b5c00 thru 0x96c3c35 (0xe036 bytes) 2024-09-19T21:36:39.504 INFO:teuthology.orchestra.run.smithi122.stdout:5589 trunc from 0xcef5e00 to 0x548c600 2024-09-19T21:36:39.518 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.518 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:39.554 INFO:teuthology.orchestra.run.smithi122.stdout:5590 trunc from 0x548c600 to 0x84ecc00 2024-09-19T21:36:39.567 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:39.563+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:39.575 INFO:teuthology.orchestra.run.smithi122.stdout:5594 read 0xfe7a00 thru 0xff59cb (0xdfcc bytes) 2024-09-19T21:36:39.578 INFO:teuthology.orchestra.run.smithi122.stdout:5595 write 0x6197a00 thru 0x61a37be (0xbdbf bytes) 2024-09-19T21:36:39.578 INFO:teuthology.orchestra.run.smithi122.stdout:5597 read 0x2df3e00 thru 0x2e0030e (0xc50f bytes) 2024-09-19T21:36:39.578 INFO:teuthology.orchestra.run.smithi122.stdout:5599 write 0x6a32600 thru 0x6a3e4de (0xbedf bytes) 2024-09-19T21:36:39.578 INFO:teuthology.orchestra.run.smithi122.stdout:5600 read 0x48da200 thru 0x48ddf26 (0x3d27 bytes) 2024-09-19T21:36:39.584 INFO:teuthology.orchestra.run.smithi122.stdout:5603 trunc from 0x84ecc00 to 0xadfa200 2024-09-19T21:36:39.615 INFO:teuthology.orchestra.run.smithi122.stdout:5605 punch from 0x46db600 to 0x46e9600, (0xe000 bytes) 2024-09-19T21:36:39.617 INFO:teuthology.orchestra.run.smithi122.stdout:5606 read 0xa59b200 thru 0xa59e1ac (0x2fad bytes) 2024-09-19T21:36:39.640 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.640 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.648 INFO:teuthology.orchestra.run.smithi122.stdout:5607 write 0xbe73c00 thru 0xbe7cb30 (0x8f31 bytes) 2024-09-19T21:36:39.649 INFO:teuthology.orchestra.run.smithi122.stdout:5608 read 0x3986c00 thru 0x398de1f (0x7220 bytes) 2024-09-19T21:36:39.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:39.667 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:39.663+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:39.676 INFO:teuthology.orchestra.run.smithi122.stdout:5609 write 0xc185800 thru 0xc193b95 (0xe396 bytes) 2024-09-19T21:36:39.678 INFO:teuthology.orchestra.run.smithi122.stdout:5610 write 0x783e000 thru 0x784c0fc (0xe0fd bytes) 2024-09-19T21:36:39.678 INFO:teuthology.orchestra.run.smithi122.stdout:5613 read 0x4f4f400 thru 0x4f5edc0 (0xf9c1 bytes) 2024-09-19T21:36:39.678 INFO:teuthology.orchestra.run.smithi122.stdout:5614 read 0x99b8e00 thru 0x99c6ea0 (0xe0a1 bytes) 2024-09-19T21:36:39.678 INFO:teuthology.orchestra.run.smithi122.stdout:5616 write 0x360bc00 thru 0x3619c38 (0xe039 bytes) 2024-09-19T21:36:39.715 INFO:teuthology.orchestra.run.smithi122.stdout:5618 write 0xc933000 thru 0xc938293 (0x5294 bytes) 2024-09-19T21:36:39.729 INFO:teuthology.orchestra.run.smithi122.stdout:5624 trunc from 0xc938400 to 0x724ba00 2024-09-19T21:36:39.767 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.768 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.768 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:39.763+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:39.770 INFO:teuthology.orchestra.run.smithi122.stdout:5625 read 0x4648c00 thru 0x4658abf (0xfec0 bytes) 2024-09-19T21:36:39.773 INFO:teuthology.orchestra.run.smithi122.stdout:5627 write 0x39f7600 thru 0x39fe750 (0x7151 bytes) 2024-09-19T21:36:39.773 INFO:teuthology.orchestra.run.smithi122.stdout:5630 read 0x4d26600 thru 0x4d31d85 (0xb786 bytes) 2024-09-19T21:36:39.773 INFO:teuthology.orchestra.run.smithi122.stdout:5631 read 0x2ab1a00 thru 0x2ab2940 (0xf41 bytes) 2024-09-19T21:36:39.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:39.797 INFO:teuthology.orchestra.run.smithi122.stdout:5633 trunc from 0x724ba00 to 0x24c000 2024-09-19T21:36:39.849 INFO:teuthology.orchestra.run.smithi122.stdout:5635 read 0x1f7800 thru 0x1fc15e (0x495f bytes) 2024-09-19T21:36:39.852 INFO:teuthology.orchestra.run.smithi122.stdout:5637 read 0x3fe00 thru 0x462e0 (0x64e1 bytes) 2024-09-19T21:36:39.853 INFO:teuthology.orchestra.run.smithi122.stdout:5639 punch from 0x68c00 to 0x6ce00, (0x4200 bytes) 2024-09-19T21:36:39.853 INFO:teuthology.orchestra.run.smithi122.stdout:5640 write 0x101800 thru 0x10b802 (0xa003 bytes) 2024-09-19T21:36:39.869 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:39.863+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:39.890 INFO:teuthology.orchestra.run.smithi122.stdout:5641 write 0x3513400 thru 0x351ad57 (0x7958 bytes) 2024-09-19T21:36:39.895 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:39.895 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:39.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:39.906 INFO:teuthology.orchestra.run.smithi122.stdout:5642 trunc from 0x351ae00 to 0x9ff8800 2024-09-19T21:36:39.947 INFO:teuthology.orchestra.run.smithi122.stdout:5643 trunc from 0x9ff8800 to 0x3278c00 2024-09-19T21:36:39.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:39.969 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:39.963+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:40.017 INFO:teuthology.orchestra.run.smithi122.stdout:5644 write 0xd7fcc00 thru 0xd80c0c4 (0xf4c5 bytes) 2024-09-19T21:36:40.038 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:40.038 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:40.039 INFO:teuthology.orchestra.run.smithi122.stdout:5645 trunc from 0xd80c200 to 0x1930000 2024-09-19T21:36:40.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:40.070 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:40.067+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:40.085 INFO:teuthology.orchestra.run.smithi122.stdout:5646 trunc from 0x1930000 to 0x3080e00 2024-09-19T21:36:40.103 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:40.126 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:40.143 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:36:40.143 INFO:teuthology.orchestra.run.smithi122.stdout:5647 write 0xc1aac00 thru 0xc1b870d (0xdb0e bytes) 2024-09-19T21:36:40.145 INFO:teuthology.orchestra.run.smithi122.stdout:5649 read 0x88a400 thru 0x891426 (0x7027 bytes) 2024-09-19T21:36:40.146 INFO:teuthology.orchestra.run.smithi122.stdout:5651 write 0x8419000 thru 0x841c086 (0x3087 bytes) 2024-09-19T21:36:40.146 INFO:teuthology.orchestra.run.smithi122.stdout:5653 read 0x90f1c00 thru 0x9101095 (0xf496 bytes) 2024-09-19T21:36:40.146 INFO:teuthology.orchestra.run.smithi122.stdout:5654 read 0x5521e00 thru 0x553018e (0xe38f bytes) 2024-09-19T21:36:40.153 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:40.158 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:40.158 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:40.159 INFO:teuthology.orchestra.run.smithi122.stdout:5655 trunc from 0xc1b8800 to 0x69f8c00 2024-09-19T21:36:40.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:40.171 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:40.167+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:40.176 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:40.198 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:40.218 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:40.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:40.239 INFO:teuthology.orchestra.run.smithi122.stdout:5657 write 0xeceda00 thru 0xecfd783 (0xfd84 bytes) 2024-09-19T21:36:40.240 INFO:teuthology.orchestra.run.smithi122.stdout:5662 read 0x233a800 thru 0x234526b (0xaa6c bytes) 2024-09-19T21:36:40.242 INFO:teuthology.orchestra.run.smithi122.stdout:5663 trunc from 0xecfd800 to 0xe3f1c00 2024-09-19T21:36:40.274 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:40.271+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:40.278 INFO:teuthology.orchestra.run.smithi122.stdout:5665 write 0x4501e00 thru 0x45087c9 (0x69ca bytes) 2024-09-19T21:36:40.281 INFO:teuthology.orchestra.run.smithi122.stdout:5668 write 0xcc3da00 thru 0xcc48070 (0xa671 bytes) 2024-09-19T21:36:40.281 INFO:teuthology.orchestra.run.smithi122.stdout:5671 punch from 0x894dc00 to 0x8951400, (0x3800 bytes) 2024-09-19T21:36:40.282 INFO:teuthology.orchestra.run.smithi122.stdout:5673 punch from 0x181c400 to 0x1824a00, (0x8600 bytes) 2024-09-19T21:36:40.282 INFO:teuthology.orchestra.run.smithi122.stdout:5674 read 0x448a600 thru 0x448b0f8 (0xaf9 bytes) 2024-09-19T21:36:40.282 INFO:teuthology.orchestra.run.smithi122.stdout:5675 punch from 0xe36ba00 to 0xe36e200, (0x2800 bytes) 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031601898000005, 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:40.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:40.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031588358999997, 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:40.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031562618999999, 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:40.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031548404999995, 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:40.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031527487999995, 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:40.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:40.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031512112000001, 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:40.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031497184000003, 2024-09-19T21:36:40.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:40.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.323 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031444563999997, 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:40.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031394985999995, 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:40.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031377770999995, 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:40.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:40.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031358632999996, 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:40.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:40.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031343785000004, 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.031320792000002, 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.348 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:40.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.030588499999993, 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.353 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.354 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:40.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.357 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.030538190000001, 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.030475878000004, 2024-09-19T21:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:40.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:40.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.029145244999995, 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:40.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.029009666999997, 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:40.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.028932908000002, 2024-09-19T21:36:40.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 86.028874184000003, 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.387 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:40.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5676 trunc from 0xe3f1c00 to 0x6292e00 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5678 punch from 0x5b0b800 to 0x5b1a400, (0xec00 bytes) 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5679 read 0x5c84e00 thru 0x5c85f18 (0x1119 bytes) 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5680 read 0x1a2d200 thru 0x1a3a1ea (0xcfeb bytes) 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5681 read 0x1900e00 thru 0x19023e5 (0x15e6 bytes) 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5682 read 0x3950e00 thru 0x395ec34 (0xde35 bytes) 2024-09-19T21:36:40.390 INFO:teuthology.orchestra.run.smithi122.stdout:5683 trunc from 0x6292e00 to 0x2fa2600 2024-09-19T21:36:40.391 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:40.383+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:40.391 INFO:teuthology.orchestra.run.smithi122.stdout:5685 write 0x2ff1600 thru 0x2ff9c7e (0x867f bytes) 2024-09-19T21:36:40.446 INFO:teuthology.orchestra.run.smithi122.stdout:5687 write 0xd95f400 thru 0xd96ef61 (0xfb62 bytes) 2024-09-19T21:36:40.448 INFO:teuthology.orchestra.run.smithi122.stdout:5689 clone 37 order 22 su 32768 sc 10 2024-09-19T21:36:40.489 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:40.483+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:40.507 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:40.507 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:40.507 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:40.507 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:40.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:40.590 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:40.587+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:40.618 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:40.618 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:40.618 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:40.619 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:40.619 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:40.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:40.691 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:40.691+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.585333040999998, 2024-09-19T21:36:40.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.584753129999996, 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:40.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:40.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.584690483000003, 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.803 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.584624163000001, 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:40.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583920484999993, 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.813 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583847663, 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:40.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.821 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583833924999993, 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.822 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:40.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:40.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583816236000004, 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:40.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583765850000006, 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:40.838 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583738960999995, 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:40.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:40.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583698385000005, 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583676771, 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:40.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:40.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.854 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.583657572000007, 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:40.860 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.09433997, 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.861 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.864 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.093716487999998, 2024-09-19T21:36:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:40.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:40.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.093384211999997, 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:40.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.876 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 4.0042929950000001, 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:40.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:36:40.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.98169664, 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:36:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:36:40.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9813659729999999, 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:36:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:36:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.980846401, 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:36:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:36:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:36:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:40.901 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:40.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:40.904 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:40.899+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:41.003 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:40.999+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.018 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:41.018 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:41.018 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:41.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:41.104 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.099+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.134 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:41.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:41.134 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:41.134 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:41.134 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:41.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:41.196 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:41.196 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:41.196 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-19T21:36:41.196 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:36:41.196 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:36:41.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-19T21:36:41.204 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:41.199+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.257 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 105.503631758, 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:41.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:41.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.263 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876292362000001, 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876239896000001, 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:41.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876209349, 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.273 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:41.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876186752000002, 2024-09-19T21:36:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:41.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876164299999999, 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876150004000003, 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.289 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:41.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876133377000002, 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876114741000002, 2024-09-19T21:36:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:41.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:41.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:41.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876088076000002, 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.300 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876077864999999, 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.876065379000003, 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872190789000001, 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:41.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872173269999998, 2024-09-19T21:36:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:41.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.321 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872135716000003, 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.326 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872117873000001, 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872105423999997, 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:41.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872094732000001, 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.872083154999999, 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:41.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:41.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:41.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 60.456111198999999, 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:41.346 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:41.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:41.350 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:41.352 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:41.352 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "" 2024-09-19T21:36:41.352 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:41.353 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.347+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:41.453 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:41.451+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:41.466 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:41.466 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:41.466 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:41.466 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:41.477 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:41.554 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.547+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.580 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:41.580 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:41.580 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:41.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:41.654 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.651+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.703 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.081487186000004, 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.707 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.081465457999997, 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:41.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.081444726000001, 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.712 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:41.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.081410289999994, 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:41.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:41.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.081367493000002, 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:41.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.725 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.076141982999999, 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:41.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:41.730 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.076119684999995, 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.076088830000003, 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:41.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.076046121000005, 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:41.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:41.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.076022133999999, 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:41.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.075990730000001, 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:41.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:41.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.075965564000001, 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:41.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.075920486000001, 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:41.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.075893742000005, 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.760 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:41.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.07588045, 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:41.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.075863530999996, 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:41.768 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6758767589999994, 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:36:41.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:36:41.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:36:41.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6743100559999995, 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:41.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:36:41.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:36:41.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:36:41.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6742737759999997, 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:41.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:36:41.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:41.790 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.6742537540000004, 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:41.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:36:41.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:41.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.796 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:41.796 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:41.796 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:41.797 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:41.798 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.791+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:41.898 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:41.891+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:41.945 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-19T21:36:41.960 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:41.960 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:41.960 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:41.960 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:41.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:41.999 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:41.995+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:42.097 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:42.097 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:42.097 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:42.097 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:42.097 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:42.099 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:42.095+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:42.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:42.156 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:36:42.202 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:42.199+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:42.233 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:42.274 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 146.338901545, 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:42.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:42.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:42.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:42.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019334213999997, 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:42.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:42.285 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019322447999997, 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:42.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:42.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:42.290 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019309555000007, 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019297531999996, 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:42.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019279888, 2024-09-19T21:36:42.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:42.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:42.303 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019264092, 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.304 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.308 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019252241000004, 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:42.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019183397000006, 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:42.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.317 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019155256999994, 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.318 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:42.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.019132127999995, 2024-09-19T21:36:42.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:42.326 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 70.004355508000003, 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.328 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.329 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:42.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.997019193, 2024-09-19T21:36:42.331 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:42.332 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.333 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:42.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.335 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.996998302999998, 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.336 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.337 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.338 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:42.339 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.340 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.989922762999996, 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:42.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.342 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.343 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:42.344 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.989902219000001, 2024-09-19T21:36:42.345 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.346 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:42.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:42.348 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.349 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.809548148000005, 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.350 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.351 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.352 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:42.353 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:42.354 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.809537720999998, 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:42.355 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.356 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:42.357 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.358 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.809527291999999, 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.360 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:42.361 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.362 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.809517228999994, 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:42.364 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:42.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:42.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:42.367 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:42.368 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:42.368 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:42.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:42.370 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:42.367+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:42.375 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:42.393 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:42.410 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:42.432 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:42.439 INFO:teuthology.orchestra.run.smithi122.stdout:5691 read 0x3080400 thru 0x308a0f7 (0x9cf8 bytes) 2024-09-19T21:36:42.455 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:42.470 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:42.467+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:42.479 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:42.485 INFO:teuthology.orchestra.run.smithi122.stdout:5692 trunc from 0xd96f000 to 0x51c9000 2024-09-19T21:36:42.491 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:42.491 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:42.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:42.541 INFO:teuthology.orchestra.run.smithi122.stdout:5693 read 0x3322800 thru 0x3327f9d (0x579e bytes) 2024-09-19T21:36:42.543 INFO:teuthology.orchestra.run.smithi122.stdout:5694 punch from 0x2601a00 to 0x260c400, (0xaa00 bytes) 2024-09-19T21:36:42.570 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:42.567+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:42.591 INFO:teuthology.orchestra.run.smithi122.stdout:5695 write 0xeb9aa00 thru 0xeba33a1 (0x89a2 bytes) 2024-09-19T21:36:42.591 INFO:teuthology.orchestra.run.smithi122.stdout:5696 read 0xc03c200 thru 0xc044156 (0x7f57 bytes) 2024-09-19T21:36:42.591 INFO:teuthology.orchestra.run.smithi122.stdout:5697 write 0x1400e00 thru 0x1402fdc (0x21dd bytes) 2024-09-19T21:36:42.605 INFO:teuthology.orchestra.run.smithi122.stdout:5699 trunc from 0xeba3400 to 0x8279200 2024-09-19T21:36:42.653 INFO:teuthology.orchestra.run.smithi122.stdout:5700 trunc from 0x8279200 to 0xb1fa000 2024-09-19T21:36:42.668 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:42.668 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:42.671 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:42.667+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:42.673 INFO:teuthology.orchestra.run.smithi122.stdout:5701 write 0x624000 thru 0x629d82 (0x5d83 bytes) 2024-09-19T21:36:42.675 INFO:teuthology.orchestra.run.smithi122.stdout:5704 write 0x8a8400 thru 0x8b701d (0xec1e bytes) 2024-09-19T21:36:42.676 INFO:teuthology.orchestra.run.smithi122.stdout:5705 punch from 0xae97800 to 0xaea6a00, (0xf200 bytes) 2024-09-19T21:36:42.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:42.684 INFO:teuthology.orchestra.run.smithi122.stdout:5706 trunc from 0xb1fa000 to 0x8dfc400 2024-09-19T21:36:42.716 INFO:teuthology.orchestra.run.smithi122.stdout:5707 trunc from 0x8dfc400 to 0x9bb9800 2024-09-19T21:36:42.737 INFO:teuthology.orchestra.run.smithi122.stdout:5708 write 0x8380200 thru 0x8385f8d (0x5d8e bytes) 2024-09-19T21:36:42.739 INFO:teuthology.orchestra.run.smithi122.stdout:5709 punch from 0x44fd600 to 0x4509a00, (0xc400 bytes) 2024-09-19T21:36:42.742 INFO:teuthology.orchestra.run.smithi122.stdout:5712 write 0xa7ae00 thru 0xa80c12 (0x5e13 bytes) 2024-09-19T21:36:42.745 INFO:teuthology.orchestra.run.smithi122.stdout:5713 read 0x4f8d400 thru 0x4f982a3 (0xaea4 bytes) 2024-09-19T21:36:42.745 INFO:teuthology.orchestra.run.smithi122.stdout:5715 write 0x4151200 thru 0x415feb0 (0xecb1 bytes) 2024-09-19T21:36:42.745 INFO:teuthology.orchestra.run.smithi122.stdout:5718 clone 38 order 25 su 4194304 sc 6 2024-09-19T21:36:42.773 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:42.771+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:42.789 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:42.789 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:42.799 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:42.874 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:42.871+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:42.917 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:42.917 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:42.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:42.975 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:42.971+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:43.034 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.034 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:43.076 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:43.071+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:43.146 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.146 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:43.167 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:36:43.177 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:43.175+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:43.270 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.270 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.278 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:43.275+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:43.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:43.307 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone37 2024-09-19T21:36:43.379 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:43.375+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:43.389 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.389 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:43.480 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:43.475+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:43.507 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.507 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:43.580 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:43.575+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:43.627 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.627 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:43.682 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:43.679+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:43.742 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.742 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:43.797 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:43.795+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:43.864 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.864 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:43.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:43.918 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:43.915+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:43.976 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-19T21:36:43.983 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:43.979+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:43.996 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:43.997 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:44.008 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:44.085 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:44.083+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:44.127 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:44.128 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:44.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:44.185 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:44.183+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:44.265 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:44.265 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:44.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:44.285 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:44.279+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:44.378 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:44.379 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:44.387 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:44.383+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:44.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:44.488 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:44.483+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:44.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:44.500 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:44.500 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:44.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:44.589 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:44.587+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:44.661 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:44.689 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:44.683+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:44.696 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 148.77275338199999, 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453186051000003, 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453174285000003, 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453161391999998, 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453149369000002, 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453131725000006, 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453115929000006, 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453104077999996, 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453035233999998, 2024-09-19T21:36:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.453007094, 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.452983965000001, 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:44.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:44.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:44.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.438207344999995, 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:44.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:44.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:44.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:44.766 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.430871030000006, 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:44.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.430850140000004, 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:44.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:44.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.775 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.423774600000002, 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:44.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:44.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.423754056000007, 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:44.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.243399984999996, 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:44.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:44.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.790 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.243389558000004, 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:44.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:44.794 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.243379129000004, 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:44.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:44.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.243369066, 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:44.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:44.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:44.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:44.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:44.804 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:44.808 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:44.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:44.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:44.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:44.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:44.810 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:44.810 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:44.810 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:44.810 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:44.810 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:44.811 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:44.811 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:44.799+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:44.906 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:44.903+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:44.967 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:44.967 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:44.967 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:44.967 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:44.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:44.986 INFO:teuthology.orchestra.run.smithi122.stdout:5719 read 0x9682c00 thru 0x968d389 (0xa78a bytes) 2024-09-19T21:36:44.986 INFO:teuthology.orchestra.run.smithi122.stdout:5720 read 0x8c6a800 thru 0x8c7a150 (0xf951 bytes) 2024-09-19T21:36:45.022 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:45.019+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:45.040 INFO:teuthology.orchestra.run.smithi122.stdout:5721 write 0xdd49200 thru 0xdd577ac (0xe5ad bytes) 2024-09-19T21:36:45.042 INFO:teuthology.orchestra.run.smithi122.stdout:5722 read 0xd079e00 thru 0xd082234 (0x8435 bytes) 2024-09-19T21:36:45.043 INFO:teuthology.orchestra.run.smithi122.stdout:5725 read 0x1265e00 thru 0x12690ef (0x32f0 bytes) 2024-09-19T21:36:45.093 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:45.093 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:45.093 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:45.093 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:45.094 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:45.103 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:45.104 INFO:teuthology.orchestra.run.smithi122.stdout:5728 trunc from 0xdd57800 to 0x12a9400 2024-09-19T21:36:45.107 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:45.103+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:45.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:36:45.203 INFO:teuthology.orchestra.run.smithi122.stdout:5730 read 0x93fa00 thru 0x945a2a (0x602b bytes) 2024-09-19T21:36:45.226 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:45.223+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:45.229 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.497087487, 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.895132242999999, 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894686260999997, 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894572082000003, 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894555648999997, 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894540093000003, 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894426023999998, 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894409060999998, 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894393076999997, 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894375902, 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894358703999998, 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894341873000002, 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894325352000003, 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894302748000001, 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.295 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894286360000002, 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:45.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:45.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 58.894177479, 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.300 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:45.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:45.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.215146444, 2024-09-19T21:36:45.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:45.305 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.306 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.308 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:45.309 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.215033295, 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.310 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.311 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:45.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.313 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:45.314 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.205767782000001, 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:45.315 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.316 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:45.317 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.318 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.319 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.205695357, 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:45.320 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:45.321 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:45.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.324 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:45.325 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:45.325 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:45.326 INFO:teuthology.orchestra.run.smithi122.stdout:5731 write 0xe5a8000 thru 0xe5b6a42 (0xea43 bytes) 2024-09-19T21:36:45.327 INFO:teuthology.orchestra.run.smithi122.stdout:5732 trunc from 0xe5b6c00 to 0xd281800 2024-09-19T21:36:45.327 INFO:teuthology.orchestra.run.smithi122.stdout:5734 trunc from 0xd281800 to 0xdee4200 2024-09-19T21:36:45.327 INFO:teuthology.orchestra.run.smithi122.stdout:5735 read 0x7924600 thru 0x792f777 (0xb178 bytes) 2024-09-19T21:36:45.327 INFO:teuthology.orchestra.run.smithi122.stdout:5736 punch from 0xdd76e00 to 0xdd79800, (0x2a00 bytes) 2024-09-19T21:36:45.327 INFO:teuthology.orchestra.run.smithi122.stdout:5739 read 0x902400 thru 0x90eff8 (0xcbf9 bytes) 2024-09-19T21:36:45.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:45.328 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:45.323+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:45.329 INFO:teuthology.orchestra.run.smithi122.stdout:5740 write 0xe3c3c00 thru 0xe3c5446 (0x1847 bytes) 2024-09-19T21:36:45.330 INFO:teuthology.orchestra.run.smithi122.stdout:5741 read 0xa5c800 thru 0xa63e8e (0x768f bytes) 2024-09-19T21:36:45.331 INFO:teuthology.orchestra.run.smithi122.stdout:5742 write 0x1fa6600 thru 0x1fb1bcd (0xb5ce bytes) 2024-09-19T21:36:45.356 INFO:teuthology.orchestra.run.smithi122.stdout:5743 trunc from 0xe3c5600 to 0x35e0e00 2024-09-19T21:36:45.405 INFO:teuthology.orchestra.run.smithi122.stdout:5744 read 0x1ae2400 thru 0x1aea32d (0x7f2e bytes) 2024-09-19T21:36:45.410 INFO:teuthology.orchestra.run.smithi122.stdout:5746 read 0x13d2800 thru 0x13d65d0 (0x3dd1 bytes) 2024-09-19T21:36:45.411 INFO:teuthology.orchestra.run.smithi122.stdout:5747 punch from 0x2b14400 to 0x2b15600, (0x1200 bytes) 2024-09-19T21:36:45.411 INFO:teuthology.orchestra.run.smithi122.stdout:5751 punch from 0xc5a400 to 0xc66400, (0xc000 bytes) 2024-09-19T21:36:45.411 INFO:teuthology.orchestra.run.smithi122.stdout:5753 punch from 0x90fa00 to 0x910e00, (0x1400 bytes) 2024-09-19T21:36:45.426 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:45.423+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:45.427 INFO:teuthology.orchestra.run.smithi122.stdout:5756 trunc from 0x35e0e00 to 0xa383a00 2024-09-19T21:36:45.445 INFO:teuthology.orchestra.run.smithi122.stdout:5757 read 0x1fba00 thru 0x206ee5 (0xb4e6 bytes) 2024-09-19T21:36:45.449 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:45.449 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:45.449 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:45.452 INFO:teuthology.orchestra.run.smithi122.stdout:5759 read 0x3a9c00 thru 0x3ab22f (0x1630 bytes) 2024-09-19T21:36:45.454 INFO:teuthology.orchestra.run.smithi122.stdout:5760 write 0x7b83000 thru 0x7b8f143 (0xc144 bytes) 2024-09-19T21:36:45.455 INFO:teuthology.orchestra.run.smithi122.stdout:5762 write 0xa2bb400 thru 0xa2c653f (0xb140 bytes) 2024-09-19T21:36:45.456 INFO:teuthology.orchestra.run.smithi122.stdout:5763 punch from 0x3aea600 to 0x3af3c00, (0x9600 bytes) 2024-09-19T21:36:45.458 INFO:teuthology.orchestra.run.smithi122.stdout:5765 write 0x2a2600 thru 0x2af004 (0xca05 bytes) 2024-09-19T21:36:45.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:45.470 INFO:teuthology.orchestra.run.smithi122.stdout:5766 trunc from 0xa383a00 to 0x68db600 2024-09-19T21:36:45.510 INFO:teuthology.orchestra.run.smithi122.stdout:5768 trunc from 0x68db600 to 0x13dd200 2024-09-19T21:36:45.527 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:45.523+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:45.540 INFO:teuthology.orchestra.run.smithi122.stdout:5770 read 0xb68400 thru 0xb68777 (0x378 bytes) 2024-09-19T21:36:45.542 INFO:teuthology.orchestra.run.smithi122.stdout:5771 read 0x13b0000 thru 0x13b4502 (0x4503 bytes) 2024-09-19T21:36:45.543 INFO:teuthology.orchestra.run.smithi122.stdout:5772 read 0xab5800 thru 0xab5b24 (0x325 bytes) 2024-09-19T21:36:45.544 INFO:teuthology.orchestra.run.smithi122.stdout:5774 punch from 0x368e00 to 0x377a00, (0xec00 bytes) 2024-09-19T21:36:45.544 INFO:teuthology.orchestra.run.smithi122.stdout:5775 read 0x11e9a00 thru 0x11f3ac6 (0xa0c7 bytes) 2024-09-19T21:36:45.563 INFO:teuthology.orchestra.run.smithi122.stdout:5776 write 0x26b4e00 thru 0x26bb27f (0x6480 bytes) 2024-09-19T21:36:45.565 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:45.565 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:45.565 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:45.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:45.566 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:45.566 INFO:teuthology.orchestra.run.smithi122.stdout:5777 read 0x8ea400 thru 0x8f9d13 (0xf914 bytes) 2024-09-19T21:36:45.567 INFO:teuthology.orchestra.run.smithi122.stdout:5781 read 0xeaaa00 thru 0xeb9cf1 (0xf2f2 bytes) 2024-09-19T21:36:45.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:45.615 INFO:teuthology.orchestra.run.smithi122.stdout:5783 write 0xe399e00 thru 0xe3a2625 (0x8826 bytes) 2024-09-19T21:36:45.629 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:45.623+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:45.638 INFO:teuthology.orchestra.run.smithi122.stdout:5784 trunc from 0xe3a2800 to 0x1b9da00 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428098328000004, 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428084788999996, 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.691 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428059048999998, 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:45.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428044834999994, 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428023917999994, 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:45.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.709 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.428008542000001, 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:45.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427993614000002, 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.719 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427940993999997, 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427891415999994, 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:45.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427874200999995, 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427855062999996, 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:45.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427840215000003, 2024-09-19T21:36:45.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427817222000002, 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427084930000007, 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:45.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:45.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:45.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.427034620000001, 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:45.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:45.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.426972308000003, 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:45.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.425641674999994, 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.759 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:45.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.762 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.425506096999996, 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.425429338000001, 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:45.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.769 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.770 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:45.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.425370614000002, 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:45.773 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.774 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.775 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:45.776 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:45.778 INFO:teuthology.orchestra.run.smithi122.stdout:5786 read 0x1260400 thru 0x126f5ec (0xf1ed bytes) 2024-09-19T21:36:45.778 INFO:teuthology.orchestra.run.smithi122.stdout:5788 write 0x4c65000 thru 0x4c69ad0 (0x4ad1 bytes) 2024-09-19T21:36:45.778 INFO:teuthology.orchestra.run.smithi122.stdout:5789 punch from 0x3e15800 to 0x3e16800, (0x1000 bytes) 2024-09-19T21:36:45.778 INFO:teuthology.orchestra.run.smithi122.stdout:5792 write 0x5fb8200 thru 0x5fbe0b1 (0x5eb2 bytes) 2024-09-19T21:36:45.779 INFO:teuthology.orchestra.run.smithi122.stdout:5793 trunc from 0x5fbe200 to 0x1f55a00 2024-09-19T21:36:45.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:45.779 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:45.771+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:45.798 INFO:teuthology.orchestra.run.smithi122.stdout:5794 punch from 0x1480600 to 0x148be00, (0xb800 bytes) 2024-09-19T21:36:45.799 INFO:teuthology.orchestra.run.smithi122.stdout:5797 read 0x18d2c00 thru 0x18d80f0 (0x54f1 bytes) 2024-09-19T21:36:45.802 INFO:teuthology.orchestra.run.smithi122.stdout:5801 read 0xb10200 thru 0xb14d29 (0x4b2a bytes) 2024-09-19T21:36:45.803 INFO:teuthology.orchestra.run.smithi122.stdout:5803 read 0x530800 thru 0x533aec (0x32ed bytes) 2024-09-19T21:36:45.831 INFO:teuthology.orchestra.run.smithi122.stdout:5805 trunc from 0x1f55a00 to 0xe887200 2024-09-19T21:36:45.853 INFO:teuthology.orchestra.run.smithi122.stdout:5806 clone 39 order 19 su 65536 sc 7 2024-09-19T21:36:45.878 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:45.875+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:45.897 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:45.897 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:45.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:45.979 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:45.975+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:46.011 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:46.011 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:46.011 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:46.011 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:46.012 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:46.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:46.080 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:46.079+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:46.130 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:46.130 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:46.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.929767537999993, 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:46.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.929187627000005, 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.929124979999997, 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:46.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.929058659999995, 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:46.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:46.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928354982000002, 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:46.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928282159999995, 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928268422000002, 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928250732999999, 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928200347000001, 2024-09-19T21:36:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928173458000003, 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928132882, 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928111267999995, 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.928092069000002, 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.438774467000002, 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.438150984999993, 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.437818708999998, 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.3487274920000001, 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:36:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:36:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.3261311370000008, 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:36:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:36:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.3258004700000008, 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:36:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.3252808980000008, 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:36:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.232 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:36:46.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:46.236 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:46.227+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:46.332 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone38 2024-09-19T21:36:46.333 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:46.327+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:46.353 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:46.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-19T21:36:46.354 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:46.354 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:46.354 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:46.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:46.435 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:46.431+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:46.467 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:46.467 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:46.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:46.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:46.531+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:46.591 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.839403342, 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:46.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:46.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.212063946000001, 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:46.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.212011480000001, 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:46.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:46.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211980933000007, 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:46.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211958335999995, 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:46.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:46.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211935883999999, 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211921587999996, 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211904961000002, 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:46.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:46.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211886324999995, 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:46.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211859660000002, 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.640 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:46.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211849448999999, 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:46.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.211836962999996, 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207962373000001, 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.658 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207944854000004, 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207907300000002, 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207889456999993, 2024-09-19T21:36:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207877007999997, 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207866315999993, 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 66.207854738999998, 2024-09-19T21:36:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 65.791882783000005, 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:46.702 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:46.703 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:46.704 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:46.699+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:46.804 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:46.799+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:46.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:46.819 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:46.819 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:46.819 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:46.819 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:46.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:46.905 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:46.899+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:46.951 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:46.977 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:46.977 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:46.978 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:46.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:46.998 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:47.006 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:47.003+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:47.021 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:47.046 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-19T21:36:47.053 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:47.081 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:47.083 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:47.084 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:47.101 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.101 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.108 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:47.103+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:47.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:47.197 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-09-19T21:36:47.209 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:47.203+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:47.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-19T21:36:47.293 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.293 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:47.308 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:47.303+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:47.408 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.408 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.409 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:47.403+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:47.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:47.510 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:47.507+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:47.526 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.526 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:47.611 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:47.607+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:47.661 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.661 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:47.718 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:47.714+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:47.774 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.774 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:47.813 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:47.806+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:47.885 INFO:teuthology.orchestra.run.smithi122.stdout:5807 punch from 0xd206a00 to 0xd206e00, (0x400 bytes) 2024-09-19T21:36:47.888 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:47.889 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:47.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:47.904 INFO:teuthology.orchestra.run.smithi122.stdout:5808 write 0x5cbc200 thru 0x5cbd309 (0x110a bytes) 2024-09-19T21:36:47.905 INFO:teuthology.orchestra.run.smithi122.stdout:5814 punch from 0xa164000 to 0xa16ea00, (0xaa00 bytes) 2024-09-19T21:36:47.914 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:47.910+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:47.924 INFO:teuthology.orchestra.run.smithi122.stdout:5815 read 0x669f800 thru 0x66af1e6 (0xf9e7 bytes) 2024-09-19T21:36:47.926 INFO:teuthology.orchestra.run.smithi122.stdout:5816 punch from 0x146ba00 to 0x1479e00, (0xe400 bytes) 2024-09-19T21:36:47.938 INFO:teuthology.orchestra.run.smithi122.stdout:5818 write 0xb723000 thru 0xb725759 (0x275a bytes) 2024-09-19T21:36:48.016 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:48.014+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:48.019 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.020 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.020 INFO:teuthology.orchestra.run.smithi122.stdout:5819 trunc from 0xe887200 to 0x113d200 2024-09-19T21:36:48.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:48.083 INFO:teuthology.orchestra.run.smithi122.stdout:5821 punch from 0xddfc00 to 0xde6c00, (0x7000 bytes) 2024-09-19T21:36:48.086 INFO:teuthology.orchestra.run.smithi122.stdout:5822 read 0xdb5000 thru 0xdc1860 (0xc861 bytes) 2024-09-19T21:36:48.117 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:48.115+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:48.120 INFO:teuthology.orchestra.run.smithi122.stdout:5823 write 0x4c43600 thru 0x4c47d0f (0x4710 bytes) 2024-09-19T21:36:48.121 INFO:teuthology.orchestra.run.smithi122.stdout:5824 punch from 0xba7600 to 0xbb3200, (0xbc00 bytes) 2024-09-19T21:36:48.121 INFO:teuthology.orchestra.run.smithi122.stdout:5826 read 0x814200 thru 0x8215fa (0xd3fb bytes) 2024-09-19T21:36:48.125 INFO:teuthology.orchestra.run.smithi122.stdout:5828 read 0x1d2600 thru 0x1e00c3 (0xdac4 bytes) 2024-09-19T21:36:48.130 INFO:teuthology.orchestra.run.smithi122.stdout:5829 read 0x777200 thru 0x779486 (0x2287 bytes) 2024-09-19T21:36:48.134 INFO:teuthology.orchestra.run.smithi122.stdout:5836 read 0x286e000 thru 0x287a22c (0xc22d bytes) 2024-09-19T21:36:48.134 INFO:teuthology.orchestra.run.smithi122.stdout:5837 punch from 0x37dbc00 to 0x37dc000, (0x400 bytes) 2024-09-19T21:36:48.134 INFO:teuthology.orchestra.run.smithi122.stdout:5838 punch from 0x3606600 to 0x3614400, (0xde00 bytes) 2024-09-19T21:36:48.134 INFO:teuthology.orchestra.run.smithi122.stdout:5839 punch from 0x330ca00 to 0x3310000, (0x3600 bytes) 2024-09-19T21:36:48.135 INFO:teuthology.orchestra.run.smithi122.stdout:5841 read 0x3b06400 thru 0x3b07e3b (0x1a3c bytes) 2024-09-19T21:36:48.150 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.151 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:48.162 INFO:teuthology.orchestra.run.smithi122.stdout:5843 write 0x550f600 thru 0x5510771 (0x1172 bytes) 2024-09-19T21:36:48.164 INFO:teuthology.orchestra.run.smithi122.stdout:5846 read 0x7b3e00 thru 0x7b9f1c (0x611d bytes) 2024-09-19T21:36:48.171 INFO:teuthology.orchestra.run.smithi122.stdout:5847 read 0x26b3600 thru 0x26bcd53 (0x9754 bytes) 2024-09-19T21:36:48.171 INFO:teuthology.orchestra.run.smithi122.stdout:5848 punch from 0x320200 to 0x328200, (0x8000 bytes) 2024-09-19T21:36:48.191 INFO:teuthology.orchestra.run.smithi122.stdout:5849 trunc from 0x5510800 to 0x1c2a00 2024-09-19T21:36:48.197 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-09-19T21:36:48.213 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-19T21:36:48.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:36:48.259 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:48.254+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:48.275 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.275 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.278 INFO:teuthology.orchestra.run.smithi122.stdout:5855 write 0x3afa000 thru 0x3afe43c (0x443d bytes) 2024-09-19T21:36:48.279 INFO:teuthology.orchestra.run.smithi122.stdout:5856 read 0x18d8a00 thru 0x18df8c2 (0x6ec3 bytes) 2024-09-19T21:36:48.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:48.318 INFO:teuthology.orchestra.run.smithi122.stdout:5857 write 0x8732c00 thru 0x87392e2 (0x66e3 bytes) 2024-09-19T21:36:48.334 INFO:teuthology.orchestra.run.smithi122.stdout:5861 trunc from 0x8739400 to 0xe50cc00 2024-09-19T21:36:48.337 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:48.335+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:48.360 INFO:teuthology.orchestra.run.smithi122.stdout:5864 punch from 0xc5fea00 to 0xc603200, (0x4800 bytes) 2024-09-19T21:36:48.362 INFO:teuthology.orchestra.run.smithi122.stdout:5865 write 0xd9c9800 thru 0xd9d57fa (0xbffb bytes) 2024-09-19T21:36:48.363 INFO:teuthology.orchestra.run.smithi122.stdout:5866 write 0x412c00 thru 0x414505 (0x1906 bytes) 2024-09-19T21:36:48.363 INFO:teuthology.orchestra.run.smithi122.stdout:5867 read 0x554800 thru 0x555b37 (0x1338 bytes) 2024-09-19T21:36:48.363 INFO:teuthology.orchestra.run.smithi122.stdout:5869 read 0xc658e00 thru 0xc65a169 (0x136a bytes) 2024-09-19T21:36:48.363 INFO:teuthology.orchestra.run.smithi122.stdout:5870 write 0x346ca00 thru 0x3478fd8 (0xc5d9 bytes) 2024-09-19T21:36:48.388 INFO:teuthology.orchestra.run.smithi122.stdout:5872 trunc from 0xe50cc00 to 0x27ad800 2024-09-19T21:36:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.407 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.419 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:48.420 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:48.414+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:48.432 INFO:teuthology.orchestra.run.smithi122.stdout:5875 read 0x5de00 thru 0x5e37c (0x57d bytes) 2024-09-19T21:36:48.435 INFO:teuthology.orchestra.run.smithi122.stdout:5876 punch from 0x1c5f000 to 0x1c61400, (0x2400 bytes) 2024-09-19T21:36:48.478 INFO:teuthology.orchestra.run.smithi122.stdout:5878 write 0xd115c00 thru 0xd11d789 (0x7b8a bytes) 2024-09-19T21:36:48.480 INFO:teuthology.orchestra.run.smithi122.stdout:5879 write 0x9a13600 thru 0x9a20cd7 (0xd6d8 bytes) 2024-09-19T21:36:48.480 INFO:teuthology.orchestra.run.smithi122.stdout:5881 write 0xc77b800 thru 0xc785f06 (0xa707 bytes) 2024-09-19T21:36:48.485 INFO:teuthology.orchestra.run.smithi122.stdout:5882 trunc from 0xd11d800 to 0xeadac00 2024-09-19T21:36:48.520 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:48.514+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:48.530 INFO:teuthology.orchestra.run.smithi122.stdout:5887 trunc from 0xeadac00 to 0x9552c00 2024-09-19T21:36:48.537 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.537 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:48.559 INFO:teuthology.orchestra.run.smithi122.stdout:5888 read 0x7cf5c00 thru 0x7cfe097 (0x8498 bytes) 2024-09-19T21:36:48.562 INFO:teuthology.orchestra.run.smithi122.stdout:5889 read 0x52ee400 thru 0x52fac5c (0xc85d bytes) 2024-09-19T21:36:48.562 INFO:teuthology.orchestra.run.smithi122.stdout:5890 punch from 0x6f59000 to 0x6f62400, (0x9400 bytes) 2024-09-19T21:36:48.562 INFO:teuthology.orchestra.run.smithi122.stdout:5893 clone 40 order 21 su 524288 sc 7 2024-09-19T21:36:48.621 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:48.619+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:48.649 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.649 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.658 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:48.722 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:48.718+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:48.766 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.766 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.776 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:48.823 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:48.818+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:48.888 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:48.888 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:48.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:48.923 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:48.918+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.008 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:49.008 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:49.017 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:49.062 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:49.058+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:49.126 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:49.118+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:49.147 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:49.147 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:49.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:49.210 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:36:49.211 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-09-19T21:36:49.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-19T21:36:49.226 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:49.222+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.239 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:49.259 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:49.280 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:49.307 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:49.324 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:49.325 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:49.325 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:49.325 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:49.325 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:49.327 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:49.322+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:49.333 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:49.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:49.339 INFO:teuthology.orchestra.run.smithi122.stdout:flattening image image_client.0-clone39 2024-09-19T21:36:49.359 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:49.428 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:49.422+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:49.464 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.841026184, 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:49.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:49.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.841004455999993, 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:49.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:49.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:49.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.840983723999997, 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:49.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:49.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.840949288000004, 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:49.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:49.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.482 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.840906490999998, 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.483 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:49.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.487 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835680980999996, 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.491 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835658683000005, 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:49.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835627828, 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.496 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:49.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.500 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835585119000001, 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:49.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835561131999995, 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:49.509 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835529727999997, 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:49.514 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835504561999997, 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:49.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:49.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835459483999998, 2024-09-19T21:36:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:49.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835432740000002, 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:49.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:49.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.526 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835419447999996, 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.530 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 76.835402529000007, 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:49.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:49.534 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 17.435415757000001, 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:36:49.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:36:49.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:36:49.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 17.433849054, 2024-09-19T21:36:49.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:36:49.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:36:49.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:36:49.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 17.433812774, 2024-09-19T21:36:49.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:36:49.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:49.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:49.553 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 17.433792751999999, 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:36:49.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:36:49.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.559 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.559 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.559 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:49.561 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:49.561 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:49.554+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:49.661 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:49.659+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:49.672 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:49.672 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:49.672 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:49.672 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:49.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:49.762 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:49.758+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.796 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:49.796 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:49.796 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:49.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:49.863 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:49.858+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:49.924 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 153.98837498899999, 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:49.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:49.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:49.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:49.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668807658000006, 2024-09-19T21:36:49.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:49.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668795892000006, 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:49.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668782999000001, 2024-09-19T21:36:49.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:49.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:49.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668770976000005, 2024-09-19T21:36:49.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:49.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.949 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668753331999994, 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:49.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:49.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.954 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668737535999995, 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:49.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:49.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668725684999998, 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:49.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:49.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668656841000001, 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:49.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668628701000003, 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:49.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.668605572000004, 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.976 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.653828951999998, 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:49.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:49.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.646492636999994, 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:49.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:49.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.646471747000007, 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.639396207000004, 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:49.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.639375662999996, 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:49.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:49.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.459021591999999, 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:49.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.459011165000007, 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.003 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:50.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:50.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:50.007 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.459000735999993, 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.458990673000002, 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:50.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.017 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:50.018 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-19T21:36:50.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:50.019 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:50.015+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:50.119 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:50.115+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:50.133 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:50.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:50.133 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:50.133 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:50.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:50.220 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:50.214+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:50.231 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-09-19T21:36:50.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-19T21:36:50.264 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:50.264 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:50.264 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:50.264 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:50.264 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:50.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:50.322 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:50.318+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:50.421 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:50.414+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:50.464 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.73344883199999, 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:50.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:50.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:50.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.131493587999998, 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.475 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.131047605999996, 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:50.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130933427000002, 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.480 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130916994000003, 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:50.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:50.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130901437999995, 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:50.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:50.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:50.493 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130787369000004, 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:50.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:50.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130770405999996, 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130754421999995, 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:50.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:50.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.506 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130737246999999, 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:50.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:50.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130720049000004, 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:50.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:50.515 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130703217999994, 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130686697000002, 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.520 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:50.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:50.524 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130664092999993, 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.525 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:50.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:50.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130647705000001, 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.130538823999998, 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:50.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.451507789000001, 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.45139464, 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:50.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:50.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:50.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:50.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.442129127000001, 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:50.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:50.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.442056701999999, 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:50.552 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:50.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:50.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:50.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:50.557 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:50.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:50.559 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:50.555+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:50.659 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:50.655+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:50.678 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:50.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:50.761 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:50.759+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:50.793 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:50.794 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:50.794 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:50.803 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:50.820 INFO:teuthology.orchestra.run.smithi122.stdout:5898 trunc from 0x9552c00 to 0xc3f4200 2024-09-19T21:36:50.850 INFO:teuthology.orchestra.run.smithi122.stdout:5899 write 0x610ce00 thru 0x6114033 (0x7234 bytes) 2024-09-19T21:36:50.851 INFO:teuthology.orchestra.run.smithi122.stdout:5900 write 0xaeaec00 thru 0xaebc158 (0xd559 bytes) 2024-09-19T21:36:50.851 INFO:teuthology.orchestra.run.smithi122.stdout:5902 clone 41 order 21 su 32768 sc 4 2024-09-19T21:36:50.861 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:50.854+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697918852000001, 2024-09-19T21:36:50.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:50.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:50.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697905313000007, 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697879572999994, 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.970 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697865359000005, 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:50.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:50.974 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697844442000005, 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697829065999997, 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:50.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.983 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697814137999998, 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697761517999993, 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:50.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:50.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697711940000005, 2024-09-19T21:36:50.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:50.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697694725000005, 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:50.998 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:50.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:51.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.002 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697675587000006, 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:51.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.007 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697660739, 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:51.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:51.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.697637745999998, 2024-09-19T21:36:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:51.016 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.696905454000003, 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:51.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.696855143999997, 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.021 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.696792832, 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.026 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:51.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.695462199000005, 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:51.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:51.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.695326621000007, 2024-09-19T21:36:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.039 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.695249861999997, 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.695191137999998, 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:51.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.049 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:51.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:51.051 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:51.046+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:51.151 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:51.147+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:51.165 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:51.166 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:51.166 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:51.166 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:51.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:51.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:36:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.2 to 1 (10000) 2024-09-19T21:36:51.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-19T21:36:51.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:36:51.252 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:51.247+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:51.330 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:51.330 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:51.330 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:51.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:51.353 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:51.346+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:51.364 INFO:teuthology.orchestra.run.smithi122.stdout:leaving image image_client.0-clone40 intact 2024-09-19T21:36:51.373 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:51.449 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:51.449 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:51.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.247205778999998, 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:51.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.454 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.246625867999995, 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:51.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.246563221000002, 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.464 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.246496901, 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245793223000007, 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245720401, 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.475 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:51.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245706662999993, 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245688974000004, 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:51.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.489 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245638588000006, 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:51.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245611698999994, 2024-09-19T21:36:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:51.496 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.497 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.498 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245571123000005, 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.499 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.501 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:51.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245549509, 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:51.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:51.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 83.245530310000007, 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.756212708000007, 2024-09-19T21:36:51.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:51.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.755589225999998, 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:51.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.755256950000003, 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.529 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.529 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:51.538 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:51.538 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.666165733, 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:36:51.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:36:51.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.643569378, 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.643238711, 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:36:51.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:36:51.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.642719139, 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:36:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:51.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:51.561 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:51.562 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:51.562 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:51.562 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:51.563 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:51.554+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:51.563 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:51.573 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:51.595 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:51.598 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:51.599 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:36:51.610 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-19T21:36:51.221416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2021875209,"num_objects":1999,"num_object_clones":545,"num_object_copies":3998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1999,"num_whiteouts":730,"num_read":285162,"num_read_kb":1018420,"num_write":16128,"num_write_kb":261550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":174875346,"num_keys_recovered":458,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14684,"ondisk_log_size":14684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":640984,"kb_used_data":388604,"kb_used_omap":7,"kb_used_meta":252280,"kb_avail":471218216,"statfs":{"total":483183820800,"available":482527453184,"internally_reserved":0,"allocated":397930496,"data_stored":519488030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":258334944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37027418,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7688,"num_read_kb":47849,"num_write":285,"num_write_kb":3646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033310"},"pg_stats":[{"pgid":"3.1f","version":"112'397","reported_seq":4246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.710444+0000","last_change":"2024-09-19T21:36:06.849962+0000","last_active":"2024-09-19T21:36:47.710444+0000","last_peered":"2024-09-19T21:36:47.710444+0000","last_clean":"2024-09-19T21:36:47.710444+0000","last_became_active":"2024-09-19T21:36:06.848734+0000","last_became_peered":"2024-09-19T21:36:06.848734+0000","last_unstale":"2024-09-19T21:36:47.710444+0000","last_undegraded":"2024-09-19T21:36:47.710444+0000","last_fullsized":"2024-09-19T21:36:47.710444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:12.838002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":44505733,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":17,"num_read":6043,"num_read_kb":38158,"num_write":376,"num_write_kb":23083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":24095839,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"109'226","reported_seq":2359,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.160351+0000","last_change":"2024-09-19T21:36:36.059553+0000","last_active":"2024-09-19T21:36:43.160351+0000","last_peered":"2024-09-19T21:36:43.160351+0000","last_clean":"2024-09-19T21:36:43.160351+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:36:43.160351+0000","last_undegraded":"2024-09-19T21:36:43.160351+0000","last_fullsized":"2024-09-19T21:36:43.160351+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":39146564,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":1748,"num_read_kb":7504,"num_write":177,"num_write_kb":3380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'1415","reported_seq":9928,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.193184+0000","last_change":"2024-09-19T21:36:38.061222+0000","last_active":"2024-09-19T21:36:47.193184+0000","last_peered":"2024-09-19T21:36:47.193184+0000","last_clean":"2024-09-19T21:36:47.193184+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:36:47.193184+0000","last_undegraded":"2024-09-19T21:36:47.193184+0000","last_fullsized":"2024-09-19T21:36:47.193184+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1415,"log_dups_size":0,"ondisk_log_size":1415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":62626163,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":16746,"num_read_kb":38790,"num_write":1855,"num_write_kb":15173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"114'438","reported_seq":15717,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190936+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:36:48.190936+0000","last_peered":"2024-09-19T21:36:48.190936+0000","last_clean":"2024-09-19T21:36:48.190936+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:36:48.190936+0000","last_undegraded":"2024-09-19T21:36:48.190936+0000","last_fullsized":"2024-09-19T21:36:48.190936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":48151604,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":16540,"num_read_kb":26027,"num_write":363,"num_write_kb":3983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"110'351","reported_seq":4143,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.181450+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:36:45.181450+0000","last_peered":"2024-09-19T21:36:45.181450+0000","last_clean":"2024-09-19T21:36:45.181450+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:36:45.181450+0000","last_undegraded":"2024-09-19T21:36:45.181450+0000","last_fullsized":"2024-09-19T21:36:45.181450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":180001481,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":24,"num_read":3996,"num_read_kb":25573,"num_write":341,"num_write_kb":5991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"114'861","reported_seq":7894,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.249356+0000","last_change":"2024-09-19T21:36:05.849867+0000","last_active":"2024-09-19T21:36:48.249356+0000","last_peered":"2024-09-19T21:36:48.249356+0000","last_clean":"2024-09-19T21:36:48.249356+0000","last_became_active":"2024-09-19T21:36:05.848564+0000","last_became_peered":"2024-09-19T21:36:05.848564+0000","last_unstale":"2024-09-19T21:36:48.249356+0000","last_undegraded":"2024-09-19T21:36:48.249356+0000","last_fullsized":"2024-09-19T21:36:48.249356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.523212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":53297386,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":19,"num_read":10778,"num_read_kb":37272,"num_write":1327,"num_write_kb":8540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4294810,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"111'349","reported_seq":14261,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.299412+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:36:47.299412+0000","last_peered":"2024-09-19T21:36:47.299412+0000","last_clean":"2024-09-19T21:36:47.299412+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:36:47.299412+0000","last_undegraded":"2024-09-19T21:36:47.299412+0000","last_fullsized":"2024-09-19T21:36:47.299412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":42093778,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":14696,"num_read_kb":33581,"num_write":433,"num_write_kb":8749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"111'259","reported_seq":9897,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.373022+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:36:45.373022+0000","last_peered":"2024-09-19T21:36:45.373022+0000","last_clean":"2024-09-19T21:36:45.373022+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:36:45.373022+0000","last_undegraded":"2024-09-19T21:36:45.373022+0000","last_fullsized":"2024-09-19T21:36:45.373022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":84483684,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":10411,"num_read_kb":41423,"num_write":331,"num_write_kb":13186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'233","reported_seq":6011,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.710961+0000","last_change":"2024-09-19T21:36:05.848894+0000","last_active":"2024-09-19T21:36:47.710961+0000","last_peered":"2024-09-19T21:36:47.710961+0000","last_clean":"2024-09-19T21:36:47.710961+0000","last_became_active":"2024-09-19T21:36:05.848182+0000","last_became_peered":"2024-09-19T21:36:05.848182+0000","last_unstale":"2024-09-19T21:36:47.710961+0000","last_undegraded":"2024-09-19T21:36:47.710961+0000","last_fullsized":"2024-09-19T21:36:47.710961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:18.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":53030400,"num_objects":75,"num_object_clones":28,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":5988,"num_read_kb":14485,"num_write":195,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'286","reported_seq":10517,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.192006+0000","last_change":"2024-09-19T21:36:35.056198+0000","last_active":"2024-09-19T21:36:48.192006+0000","last_peered":"2024-09-19T21:36:48.192006+0000","last_clean":"2024-09-19T21:36:48.192006+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:36:48.192006+0000","last_undegraded":"2024-09-19T21:36:48.192006+0000","last_fullsized":"2024-09-19T21:36:48.192006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":74823757,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":11504,"num_read_kb":29192,"num_write":237,"num_write_kb":5846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"114'325","reported_seq":6617,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.200165+0000","last_change":"2024-09-19T21:36:05.846800+0000","last_active":"2024-09-19T21:36:48.200165+0000","last_peered":"2024-09-19T21:36:48.200165+0000","last_clean":"2024-09-19T21:36:48.200165+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:36:48.200165+0000","last_undegraded":"2024-09-19T21:36:48.200165+0000","last_fullsized":"2024-09-19T21:36:48.200165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":15141667,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":6515,"num_read_kb":11953,"num_write":256,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"110'274","reported_seq":9721,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196480+0000","last_change":"2024-09-19T21:36:36.060626+0000","last_active":"2024-09-19T21:36:47.196480+0000","last_peered":"2024-09-19T21:36:47.196480+0000","last_clean":"2024-09-19T21:36:47.196480+0000","last_became_active":"2024-09-19T21:36:36.059875+0000","last_became_peered":"2024-09-19T21:36:36.059875+0000","last_unstale":"2024-09-19T21:36:47.196480+0000","last_undegraded":"2024-09-19T21:36:47.196480+0000","last_fullsized":"2024-09-19T21:36:47.196480+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:30:00.883700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":50334224,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":9582,"num_read_kb":27001,"num_write":237,"num_write_kb":4733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6472736,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196658+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:36:47.196658+0000","last_peered":"2024-09-19T21:36:47.196658+0000","last_clean":"2024-09-19T21:36:47.196658+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:36:47.196658+0000","last_undegraded":"2024-09-19T21:36:47.196658+0000","last_fullsized":"2024-09-19T21:36:47.196658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"114'435","reported_seq":8278,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190186+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:36:48.190186+0000","last_peered":"2024-09-19T21:36:48.190186+0000","last_clean":"2024-09-19T21:36:48.190186+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:36:48.190186+0000","last_undegraded":"2024-09-19T21:36:48.190186+0000","last_fullsized":"2024-09-19T21:36:48.190186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":71374466,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":8572,"num_read_kb":23369,"num_write":394,"num_write_kb":6442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.216632+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:36:47.216632+0000","last_peered":"2024-09-19T21:36:47.216632+0000","last_clean":"2024-09-19T21:36:47.216632+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:36:47.216632+0000","last_undegraded":"2024-09-19T21:36:47.216632+0000","last_fullsized":"2024-09-19T21:36:47.216632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'615","reported_seq":7917,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.248832+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:36:48.248832+0000","last_peered":"2024-09-19T21:36:48.248832+0000","last_clean":"2024-09-19T21:36:48.248832+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:36:48.248832+0000","last_undegraded":"2024-09-19T21:36:48.248832+0000","last_fullsized":"2024-09-19T21:36:48.248832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":45459706,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":9290,"num_read_kb":40977,"num_write":730,"num_write_kb":10663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196449+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:36:47.196449+0000","last_peered":"2024-09-19T21:36:47.196449+0000","last_clean":"2024-09-19T21:36:47.196449+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:36:47.196449+0000","last_undegraded":"2024-09-19T21:36:47.196449+0000","last_fullsized":"2024-09-19T21:36:47.196449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"112'429","reported_seq":12258,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.216261+0000","last_change":"2024-09-19T21:36:36.061082+0000","last_active":"2024-09-19T21:36:47.216261+0000","last_peered":"2024-09-19T21:36:47.216261+0000","last_clean":"2024-09-19T21:36:47.216261+0000","last_became_active":"2024-09-19T21:36:36.058617+0000","last_became_peered":"2024-09-19T21:36:36.058617+0000","last_unstale":"2024-09-19T21:36:47.216261+0000","last_undegraded":"2024-09-19T21:36:47.216261+0000","last_fullsized":"2024-09-19T21:36:47.216261+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:26:34.274961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":75941610,"num_objects":76,"num_object_clones":24,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":12321,"num_read_kb":41212,"num_write":336,"num_write_kb":6994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":15059605,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.159814+0000","last_change":"2024-09-19T21:36:32.030602+0000","last_active":"2024-09-19T21:36:43.159814+0000","last_peered":"2024-09-19T21:36:43.159814+0000","last_clean":"2024-09-19T21:36:43.159814+0000","last_became_active":"2024-09-19T21:36:32.030278+0000","last_became_peered":"2024-09-19T21:36:32.030278+0000","last_unstale":"2024-09-19T21:36:43.159814+0000","last_undegraded":"2024-09-19T21:36:43.159814+0000","last_fullsized":"2024-09-19T21:36:43.159814+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:24:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"113'786","reported_seq":4961,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.047514+0000","last_change":"2024-09-19T21:36:37.062328+0000","last_active":"2024-09-19T21:36:48.047514+0000","last_peered":"2024-09-19T21:36:48.047514+0000","last_clean":"2024-09-19T21:36:48.047514+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:36:48.047514+0000","last_undegraded":"2024-09-19T21:36:48.047514+0000","last_fullsized":"2024-09-19T21:36:48.047514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":45436124,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":7870,"num_read_kb":22291,"num_write":1009,"num_write_kb":8262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'709","reported_seq":8953,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199825+0000","last_change":"2024-09-19T21:36:37.062695+0000","last_active":"2024-09-19T21:36:48.199825+0000","last_peered":"2024-09-19T21:36:48.199825+0000","last_clean":"2024-09-19T21:36:48.199825+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:36:48.199825+0000","last_undegraded":"2024-09-19T21:36:48.199825+0000","last_fullsized":"2024-09-19T21:36:48.199825+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":60057909,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":12745,"num_read_kb":18492,"num_write":775,"num_write_kb":4812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":136,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.190026+0000","last_change":"2024-09-19T21:36:02.839832+0000","last_active":"2024-09-19T21:36:47.190026+0000","last_peered":"2024-09-19T21:36:47.190026+0000","last_clean":"2024-09-19T21:36:47.190026+0000","last_became_active":"2024-09-19T21:36:02.830895+0000","last_became_peered":"2024-09-19T21:36:02.830895+0000","last_unstale":"2024-09-19T21:36:47.190026+0000","last_undegraded":"2024-09-19T21:36:47.190026+0000","last_fullsized":"2024-09-19T21:36:47.190026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:40.111547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"109'450","reported_seq":11453,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.192962+0000","last_change":"2024-09-19T21:36:38.063350+0000","last_active":"2024-09-19T21:36:47.192962+0000","last_peered":"2024-09-19T21:36:47.192962+0000","last_clean":"2024-09-19T21:36:47.192962+0000","last_became_active":"2024-09-19T21:36:38.061093+0000","last_became_peered":"2024-09-19T21:36:38.061093+0000","last_unstale":"2024-09-19T21:36:47.192962+0000","last_undegraded":"2024-09-19T21:36:47.192962+0000","last_fullsized":"2024-09-19T21:36:47.192962+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:38:17.668473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":36375145,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":12585,"num_read_kb":49685,"num_write":437,"num_write_kb":12178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":17020731,"num_keys_recovered":94,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.159455+0000","last_change":"2024-09-19T21:36:02.831334+0000","last_active":"2024-09-19T21:36:43.159455+0000","last_peered":"2024-09-19T21:36:43.159455+0000","last_clean":"2024-09-19T21:36:43.159455+0000","last_became_active":"2024-09-19T21:36:02.830988+0000","last_became_peered":"2024-09-19T21:36:02.830988+0000","last_unstale":"2024-09-19T21:36:43.159455+0000","last_undegraded":"2024-09-19T21:36:43.159455+0000","last_fullsized":"2024-09-19T21:36:43.159455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:09:35.907712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"114'646","reported_seq":8162,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.206098+0000","last_change":"2024-09-19T21:36:35.058401+0000","last_active":"2024-09-19T21:36:48.206098+0000","last_peered":"2024-09-19T21:36:48.206098+0000","last_clean":"2024-09-19T21:36:48.206098+0000","last_became_active":"2024-09-19T21:36:35.056705+0000","last_became_peered":"2024-09-19T21:36:35.056705+0000","last_unstale":"2024-09-19T21:36:48.206098+0000","last_undegraded":"2024-09-19T21:36:48.206098+0000","last_fullsized":"2024-09-19T21:36:48.206098+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:15:07.237414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":88874676,"num_objects":77,"num_object_clones":24,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":28,"num_read":9298,"num_read_kb":32973,"num_write":911,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":13548106,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.192856+0000","last_change":"2024-09-19T21:36:32.033895+0000","last_active":"2024-09-19T21:36:47.192856+0000","last_peered":"2024-09-19T21:36:47.192856+0000","last_clean":"2024-09-19T21:36:47.192856+0000","last_became_active":"2024-09-19T21:36:32.033629+0000","last_became_peered":"2024-09-19T21:36:32.033629+0000","last_unstale":"2024-09-19T21:36:47.192856+0000","last_undegraded":"2024-09-19T21:36:47.192856+0000","last_fullsized":"2024-09-19T21:36:47.192856+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:52:25.528924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"110'381","reported_seq":11184,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.181587+0000","last_change":"2024-09-19T21:36:35.050437+0000","last_active":"2024-09-19T21:36:45.181587+0000","last_peered":"2024-09-19T21:36:45.181587+0000","last_clean":"2024-09-19T21:36:45.181587+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:36:45.181587+0000","last_undegraded":"2024-09-19T21:36:45.181587+0000","last_fullsized":"2024-09-19T21:36:45.181587+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":136467521,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":12084,"num_read_kb":38943,"num_write":416,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":135,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196364+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:36:47.196364+0000","last_peered":"2024-09-19T21:36:47.196364+0000","last_clean":"2024-09-19T21:36:47.196364+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:36:47.196364+0000","last_undegraded":"2024-09-19T21:36:47.196364+0000","last_fullsized":"2024-09-19T21:36:47.196364+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":321,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196177+0000","last_change":"2024-09-19T21:36:37.060633+0000","last_active":"2024-09-19T21:36:47.196177+0000","last_peered":"2024-09-19T21:36:47.196177+0000","last_clean":"2024-09-19T21:36:47.196177+0000","last_became_active":"2024-09-19T21:36:37.060036+0000","last_became_peered":"2024-09-19T21:36:37.060036+0000","last_unstale":"2024-09-19T21:36:47.196177+0000","last_undegraded":"2024-09-19T21:36:47.196177+0000","last_fullsized":"2024-09-19T21:36:47.196177+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:41:47.787667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"114'434","reported_seq":15180,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190149+0000","last_change":"2024-09-19T21:36:37.060544+0000","last_active":"2024-09-19T21:36:48.190149+0000","last_peered":"2024-09-19T21:36:48.190149+0000","last_clean":"2024-09-19T21:36:48.190149+0000","last_became_active":"2024-09-19T21:36:37.059116+0000","last_became_peered":"2024-09-19T21:36:37.059116+0000","last_unstale":"2024-09-19T21:36:48.190149+0000","last_undegraded":"2024-09-19T21:36:48.190149+0000","last_fullsized":"2024-09-19T21:36:48.190149+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:14:10.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":113242200,"num_objects":88,"num_object_clones":26,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":32,"num_read":14193,"num_read_kb":61228,"num_write":514,"num_write_kb":12983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":17942192,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"114'374","reported_seq":5417,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199547+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:36:48.199547+0000","last_peered":"2024-09-19T21:36:48.199547+0000","last_clean":"2024-09-19T21:36:48.199547+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:36:48.199547+0000","last_undegraded":"2024-09-19T21:36:48.199547+0000","last_fullsized":"2024-09-19T21:36:48.199547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":103381759,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":21,"num_read":5804,"num_read_kb":18673,"num_write":350,"num_write_kb":6828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"114'487","reported_seq":7830,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199602+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:36:48.199602+0000","last_peered":"2024-09-19T21:36:48.199602+0000","last_clean":"2024-09-19T21:36:48.199602+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:36:48.199602+0000","last_undegraded":"2024-09-19T21:36:48.199602+0000","last_fullsized":"2024-09-19T21:36:48.199602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":44002612,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":9234,"num_read_kb":47108,"num_write":584,"num_write_kb":13272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"111'255","reported_seq":2109,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-19T21:36:46.193027+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:36:46.193027+0000","last_peered":"2024-09-19T21:36:46.193027+0000","last_clean":"2024-09-19T21:36:46.193027+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:36:46.193027+0000","last_undegraded":"2024-09-19T21:36:46.193027+0000","last_fullsized":"2024-09-19T21:36:46.193027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":22356612,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":13,"num_read":2111,"num_read_kb":20155,"num_write":281,"num_write_kb":6539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"110'191","reported_seq":2988,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199892+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:36:48.199892+0000","last_peered":"2024-09-19T21:36:48.199892+0000","last_clean":"2024-09-19T21:36:48.199892+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:36:48.199892+0000","last_undegraded":"2024-09-19T21:36:48.199892+0000","last_fullsized":"2024-09-19T21:36:48.199892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":17011712,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":2522,"num_read_kb":12510,"num_write":180,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"114'396","reported_seq":9273,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.224054+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:36:48.224054+0000","last_peered":"2024-09-19T21:36:48.224054+0000","last_clean":"2024-09-19T21:36:48.224054+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:36:48.224054+0000","last_undegraded":"2024-09-19T21:36:48.224054+0000","last_fullsized":"2024-09-19T21:36:48.224054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":122207525,"num_objects":86,"num_object_clones":27,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":31,"num_read":9296,"num_read_kb":30953,"num_write":435,"num_write_kb":9593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"110'496","reported_seq":5988,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.583211+0000","last_change":"2024-09-19T21:36:36.059649+0000","last_active":"2024-09-19T21:36:47.583211+0000","last_peered":"2024-09-19T21:36:47.583211+0000","last_clean":"2024-09-19T21:36:47.583211+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:36:47.583211+0000","last_undegraded":"2024-09-19T21:36:47.583211+0000","last_fullsized":"2024-09-19T21:36:47.583211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":33208372,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":8078,"num_read_kb":50986,"num_write":422,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"114'715","reported_seq":4133,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.230389+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:36:48.230389+0000","last_peered":"2024-09-19T21:36:48.230389+0000","last_clean":"2024-09-19T21:36:48.230389+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:36:48.230389+0000","last_undegraded":"2024-09-19T21:36:48.230389+0000","last_fullsized":"2024-09-19T21:36:48.230389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":11077999,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":6760,"num_read_kb":10636,"num_write":801,"num_write_kb":2930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"114'484","reported_seq":7890,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.221884+0000","last_change":"2024-09-19T21:36:05.849347+0000","last_active":"2024-09-19T21:36:48.221884+0000","last_peered":"2024-09-19T21:36:48.221884+0000","last_clean":"2024-09-19T21:36:48.221884+0000","last_became_active":"2024-09-19T21:36:05.848208+0000","last_became_peered":"2024-09-19T21:36:05.848208+0000","last_unstale":"2024-09-19T21:36:48.221884+0000","last_undegraded":"2024-09-19T21:36:48.221884+0000","last_fullsized":"2024-09-19T21:36:48.221884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:25:57.433432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":47277705,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":35,"num_read":9577,"num_read_kb":56968,"num_write":438,"num_write_kb":16073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":13819481,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"110'311","reported_seq":3534,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.708462+0000","last_change":"2024-09-19T21:36:35.057415+0000","last_active":"2024-09-19T21:36:47.708462+0000","last_peered":"2024-09-19T21:36:47.708462+0000","last_clean":"2024-09-19T21:36:47.708462+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:36:47.708462+0000","last_undegraded":"2024-09-19T21:36:47.708462+0000","last_fullsized":"2024-09-19T21:36:47.708462+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":80246308,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":3753,"num_read_kb":19636,"num_write":236,"num_write_kb":2823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"113'229","reported_seq":6818,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.228478+0000","last_change":"2024-09-19T21:36:05.847174+0000","last_active":"2024-09-19T21:36:48.228478+0000","last_peered":"2024-09-19T21:36:48.228478+0000","last_clean":"2024-09-19T21:36:48.228478+0000","last_became_active":"2024-09-19T21:36:05.846322+0000","last_became_peered":"2024-09-19T21:36:05.846322+0000","last_unstale":"2024-09-19T21:36:48.228478+0000","last_undegraded":"2024-09-19T21:36:48.228478+0000","last_fullsized":"2024-09-19T21:36:48.228478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:34.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":84715008,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":6453,"num_read_kb":69979,"num_write":243,"num_write_kb":8535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'327","reported_seq":7492,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196241+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:36:47.196241+0000","last_peered":"2024-09-19T21:36:47.196241+0000","last_clean":"2024-09-19T21:36:47.196241+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:36:47.196241+0000","last_undegraded":"2024-09-19T21:36:47.196241+0000","last_fullsized":"2024-09-19T21:36:47.196241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":34933412,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":7953,"num_read_kb":20587,"num_write":347,"num_write_kb":5705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2021284822,"num_objects":1995,"num_object_clones":545,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":730,"num_read":285046,"num_read_kb":1018320,"num_write":15967,"num_write_kb":259132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":173104204,"num_keys_recovered":458,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446402560,"data_stored":590238924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14564,"ondisk_log_size":14564,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95588,"kb_used_data":68568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94276252,"statfs":{"total":96636764160,"available":96538882048,"internally_reserved":0,"allocated":70213632,"data_stored":90760278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88100000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0640000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.41499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.004,"5min":1.048,"15min":1.048},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.98399999999999999,"5min":0.996,"15min":0.996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.024,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.25,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2889999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123940,"kb_used_data":66580,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94247900,"statfs":{"total":96636764160,"available":96509849600,"internally_reserved":0,"allocated":68177920,"data_stored":81544465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.872},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4330000000000001,"5min":1.921,"15min":1.921},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.879,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.626,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.81699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133316,"kb_used_data":80052,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94238524,"statfs":{"total":96636764160,"available":96500248576,"internally_reserved":0,"allocated":81973248,"data_stored":112216372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":2.5169999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":3.5750000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.66700000000000004,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":3.5379999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":3.6059999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.625}]},{"osd":5,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.266,"5min":1.282,"15min":1.282},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.86599999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":3.637}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145988,"kb_used_data":84516,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94225852,"statfs":{"total":96636764160,"available":96487272448,"internally_reserved":0,"allocated":86544384,"data_stored":122423493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.96099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.87,"5min":1.333,"15min":1.333},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.012,"15min":1.012},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.63},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0900000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0740000000000001,"5min":1.718,"15min":1.718},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.93500000000000005}]}]},{"osd":3,"up_from":12,"seq":51539607586,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142152,"kb_used_data":88888,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94229688,"statfs":{"total":96636764160,"available":96491200512,"internally_reserved":0,"allocated":91021312,"data_stored":112543422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92000000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0269999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":1.175,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":3.2719999999999998,"5min":1.2330000000000001,"15min":0.89300000000000002},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":42.182000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.99399999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.105,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":85417984,"data_stored":122104692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80248832,"data_stored":111279848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66715648,"data_stored":80870173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":89706496,"data_stored":112005716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69013504,"data_stored":90348539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":55300096,"data_stored":73629956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:36:51.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:36:51.623 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:51.626 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:51.663 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:51.658+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:51.685 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:51.686 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:51.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:51.764 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:51.759+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:51.801 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:51.801 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:51.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:51.864 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:51.858+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:51.922 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:51.922 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:51.924 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:51.928 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:36:51.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:51.941 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-19T21:36:51.221416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2021875209,"num_objects":1999,"num_object_clones":545,"num_object_copies":3998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1999,"num_whiteouts":730,"num_read":285162,"num_read_kb":1018420,"num_write":16128,"num_write_kb":261550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":174875346,"num_keys_recovered":458,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14684,"ondisk_log_size":14684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":640984,"kb_used_data":388604,"kb_used_omap":7,"kb_used_meta":252280,"kb_avail":471218216,"statfs":{"total":483183820800,"available":482527453184,"internally_reserved":0,"allocated":397930496,"data_stored":519488030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":258334944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37027418,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7688,"num_read_kb":47849,"num_write":285,"num_write_kb":3646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.033310"},"pg_stats":[{"pgid":"3.1f","version":"112'397","reported_seq":4246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.710444+0000","last_change":"2024-09-19T21:36:06.849962+0000","last_active":"2024-09-19T21:36:47.710444+0000","last_peered":"2024-09-19T21:36:47.710444+0000","last_clean":"2024-09-19T21:36:47.710444+0000","last_became_active":"2024-09-19T21:36:06.848734+0000","last_became_peered":"2024-09-19T21:36:06.848734+0000","last_unstale":"2024-09-19T21:36:47.710444+0000","last_undegraded":"2024-09-19T21:36:47.710444+0000","last_fullsized":"2024-09-19T21:36:47.710444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:12.838002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048333600000000001,"stat_sum":{"num_bytes":44505733,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":17,"num_read":6043,"num_read_kb":38158,"num_write":376,"num_write_kb":23083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":24095839,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"109'226","reported_seq":2359,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.160351+0000","last_change":"2024-09-19T21:36:36.059553+0000","last_active":"2024-09-19T21:36:43.160351+0000","last_peered":"2024-09-19T21:36:43.160351+0000","last_clean":"2024-09-19T21:36:43.160351+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:36:43.160351+0000","last_undegraded":"2024-09-19T21:36:43.160351+0000","last_fullsized":"2024-09-19T21:36:43.160351+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010498809999999999,"stat_sum":{"num_bytes":39146564,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":1748,"num_read_kb":7504,"num_write":177,"num_write_kb":3380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"110'1415","reported_seq":9928,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.193184+0000","last_change":"2024-09-19T21:36:38.061222+0000","last_active":"2024-09-19T21:36:47.193184+0000","last_peered":"2024-09-19T21:36:47.193184+0000","last_clean":"2024-09-19T21:36:47.193184+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:36:47.193184+0000","last_undegraded":"2024-09-19T21:36:47.193184+0000","last_fullsized":"2024-09-19T21:36:47.193184+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1415,"log_dups_size":0,"ondisk_log_size":1415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000330355,"stat_sum":{"num_bytes":62626163,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":16746,"num_read_kb":38790,"num_write":1855,"num_write_kb":15173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"114'438","reported_seq":15717,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190936+0000","last_change":"2024-09-19T21:34:32.002185+0000","last_active":"2024-09-19T21:36:48.190936+0000","last_peered":"2024-09-19T21:36:48.190936+0000","last_clean":"2024-09-19T21:36:48.190936+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:36:48.190936+0000","last_undegraded":"2024-09-19T21:36:48.190936+0000","last_fullsized":"2024-09-19T21:36:48.190936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132047,"stat_sum":{"num_bytes":48151604,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":16540,"num_read_kb":26027,"num_write":363,"num_write_kb":3983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"110'351","reported_seq":4143,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.181450+0000","last_change":"2024-09-19T21:34:32.005728+0000","last_active":"2024-09-19T21:36:45.181450+0000","last_peered":"2024-09-19T21:36:45.181450+0000","last_clean":"2024-09-19T21:36:45.181450+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:36:45.181450+0000","last_undegraded":"2024-09-19T21:36:45.181450+0000","last_fullsized":"2024-09-19T21:36:45.181450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012947919999999999,"stat_sum":{"num_bytes":180001481,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":24,"num_read":3996,"num_read_kb":25573,"num_write":341,"num_write_kb":5991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"114'861","reported_seq":7894,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.249356+0000","last_change":"2024-09-19T21:36:05.849867+0000","last_active":"2024-09-19T21:36:48.249356+0000","last_peered":"2024-09-19T21:36:48.249356+0000","last_clean":"2024-09-19T21:36:48.249356+0000","last_became_active":"2024-09-19T21:36:05.848564+0000","last_became_peered":"2024-09-19T21:36:05.848564+0000","last_unstale":"2024-09-19T21:36:48.249356+0000","last_undegraded":"2024-09-19T21:36:48.249356+0000","last_fullsized":"2024-09-19T21:36:48.249356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.523212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045361300000000001,"stat_sum":{"num_bytes":53297386,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":19,"num_read":10778,"num_read_kb":37272,"num_write":1327,"num_write_kb":8540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4294810,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"111'349","reported_seq":14261,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.299412+0000","last_change":"2024-09-19T21:34:32.022875+0000","last_active":"2024-09-19T21:36:47.299412+0000","last_peered":"2024-09-19T21:36:47.299412+0000","last_clean":"2024-09-19T21:36:47.299412+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:36:47.299412+0000","last_undegraded":"2024-09-19T21:36:47.299412+0000","last_fullsized":"2024-09-19T21:36:47.299412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047299499999999998,"stat_sum":{"num_bytes":42093778,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":14696,"num_read_kb":33581,"num_write":433,"num_write_kb":8749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"111'259","reported_seq":9897,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.373022+0000","last_change":"2024-09-19T21:34:32.005028+0000","last_active":"2024-09-19T21:36:45.373022+0000","last_peered":"2024-09-19T21:36:45.373022+0000","last_clean":"2024-09-19T21:36:45.373022+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:36:45.373022+0000","last_undegraded":"2024-09-19T21:36:45.373022+0000","last_fullsized":"2024-09-19T21:36:45.373022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079835300000000002,"stat_sum":{"num_bytes":84483684,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":10411,"num_read_kb":41423,"num_write":331,"num_write_kb":13186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"111'233","reported_seq":6011,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.710961+0000","last_change":"2024-09-19T21:36:05.848894+0000","last_active":"2024-09-19T21:36:47.710961+0000","last_peered":"2024-09-19T21:36:47.710961+0000","last_clean":"2024-09-19T21:36:47.710961+0000","last_became_active":"2024-09-19T21:36:05.848182+0000","last_became_peered":"2024-09-19T21:36:05.848182+0000","last_unstale":"2024-09-19T21:36:47.710961+0000","last_undegraded":"2024-09-19T21:36:47.710961+0000","last_fullsized":"2024-09-19T21:36:47.710961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:18.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047273699999999998,"stat_sum":{"num_bytes":53030400,"num_objects":75,"num_object_clones":28,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":5988,"num_read_kb":14485,"num_write":195,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"114'286","reported_seq":10517,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.192006+0000","last_change":"2024-09-19T21:36:35.056198+0000","last_active":"2024-09-19T21:36:48.192006+0000","last_peered":"2024-09-19T21:36:48.192006+0000","last_clean":"2024-09-19T21:36:48.192006+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:36:48.192006+0000","last_undegraded":"2024-09-19T21:36:48.192006+0000","last_fullsized":"2024-09-19T21:36:48.192006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039759300000000001,"stat_sum":{"num_bytes":74823757,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":11504,"num_read_kb":29192,"num_write":237,"num_write_kb":5846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"114'325","reported_seq":6617,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.200165+0000","last_change":"2024-09-19T21:36:05.846800+0000","last_active":"2024-09-19T21:36:48.200165+0000","last_peered":"2024-09-19T21:36:48.200165+0000","last_clean":"2024-09-19T21:36:48.200165+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:36:48.200165+0000","last_undegraded":"2024-09-19T21:36:48.200165+0000","last_fullsized":"2024-09-19T21:36:48.200165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033998399999999999,"stat_sum":{"num_bytes":15141667,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":6515,"num_read_kb":11953,"num_write":256,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"110'274","reported_seq":9721,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196480+0000","last_change":"2024-09-19T21:36:36.060626+0000","last_active":"2024-09-19T21:36:47.196480+0000","last_peered":"2024-09-19T21:36:47.196480+0000","last_clean":"2024-09-19T21:36:47.196480+0000","last_became_active":"2024-09-19T21:36:36.059875+0000","last_became_peered":"2024-09-19T21:36:36.059875+0000","last_unstale":"2024-09-19T21:36:47.196480+0000","last_undegraded":"2024-09-19T21:36:47.196480+0000","last_fullsized":"2024-09-19T21:36:47.196480+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:30:00.883700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001345207,"stat_sum":{"num_bytes":50334224,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":9582,"num_read_kb":27001,"num_write":237,"num_write_kb":4733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6472736,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196658+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:36:47.196658+0000","last_peered":"2024-09-19T21:36:47.196658+0000","last_clean":"2024-09-19T21:36:47.196658+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:36:47.196658+0000","last_undegraded":"2024-09-19T21:36:47.196658+0000","last_fullsized":"2024-09-19T21:36:47.196658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"114'435","reported_seq":8278,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190186+0000","last_change":"2024-09-19T21:34:32.024720+0000","last_active":"2024-09-19T21:36:48.190186+0000","last_peered":"2024-09-19T21:36:48.190186+0000","last_clean":"2024-09-19T21:36:48.190186+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:36:48.190186+0000","last_undegraded":"2024-09-19T21:36:48.190186+0000","last_fullsized":"2024-09-19T21:36:48.190186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021408120000000002,"stat_sum":{"num_bytes":71374466,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":8572,"num_read_kb":23369,"num_write":394,"num_write_kb":6442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.216632+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:36:47.216632+0000","last_peered":"2024-09-19T21:36:47.216632+0000","last_clean":"2024-09-19T21:36:47.216632+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:36:47.216632+0000","last_undegraded":"2024-09-19T21:36:47.216632+0000","last_fullsized":"2024-09-19T21:36:47.216632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"114'615","reported_seq":7917,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.248832+0000","last_change":"2024-09-19T21:34:32.024685+0000","last_active":"2024-09-19T21:36:48.248832+0000","last_peered":"2024-09-19T21:36:48.248832+0000","last_clean":"2024-09-19T21:36:48.248832+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:36:48.248832+0000","last_undegraded":"2024-09-19T21:36:48.248832+0000","last_fullsized":"2024-09-19T21:36:48.248832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020543499999999999,"stat_sum":{"num_bytes":45459706,"num_objects":56,"num_object_clones":12,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":9290,"num_read_kb":40977,"num_write":730,"num_write_kb":10663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":110,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196449+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:36:47.196449+0000","last_peered":"2024-09-19T21:36:47.196449+0000","last_clean":"2024-09-19T21:36:47.196449+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:36:47.196449+0000","last_undegraded":"2024-09-19T21:36:47.196449+0000","last_fullsized":"2024-09-19T21:36:47.196449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"112'429","reported_seq":12258,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.216261+0000","last_change":"2024-09-19T21:36:36.061082+0000","last_active":"2024-09-19T21:36:47.216261+0000","last_peered":"2024-09-19T21:36:47.216261+0000","last_clean":"2024-09-19T21:36:47.216261+0000","last_became_active":"2024-09-19T21:36:36.058617+0000","last_became_peered":"2024-09-19T21:36:36.058617+0000","last_unstale":"2024-09-19T21:36:47.216261+0000","last_undegraded":"2024-09-19T21:36:47.216261+0000","last_fullsized":"2024-09-19T21:36:47.216261+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:26:34.274961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018312269999999999,"stat_sum":{"num_bytes":75941610,"num_objects":76,"num_object_clones":24,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":12321,"num_read_kb":41212,"num_write":336,"num_write_kb":6994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":15059605,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":123,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.159814+0000","last_change":"2024-09-19T21:36:32.030602+0000","last_active":"2024-09-19T21:36:43.159814+0000","last_peered":"2024-09-19T21:36:43.159814+0000","last_clean":"2024-09-19T21:36:43.159814+0000","last_became_active":"2024-09-19T21:36:32.030278+0000","last_became_peered":"2024-09-19T21:36:32.030278+0000","last_unstale":"2024-09-19T21:36:43.159814+0000","last_undegraded":"2024-09-19T21:36:43.159814+0000","last_fullsized":"2024-09-19T21:36:43.159814+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:24:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"113'786","reported_seq":4961,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.047514+0000","last_change":"2024-09-19T21:36:37.062328+0000","last_active":"2024-09-19T21:36:48.047514+0000","last_peered":"2024-09-19T21:36:48.047514+0000","last_clean":"2024-09-19T21:36:48.047514+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:36:48.047514+0000","last_undegraded":"2024-09-19T21:36:48.047514+0000","last_fullsized":"2024-09-19T21:36:48.047514+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067884,"stat_sum":{"num_bytes":45436124,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":7870,"num_read_kb":22291,"num_write":1009,"num_write_kb":8262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"114'709","reported_seq":8953,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199825+0000","last_change":"2024-09-19T21:36:37.062695+0000","last_active":"2024-09-19T21:36:48.199825+0000","last_peered":"2024-09-19T21:36:48.199825+0000","last_clean":"2024-09-19T21:36:48.199825+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:36:48.199825+0000","last_undegraded":"2024-09-19T21:36:48.199825+0000","last_fullsized":"2024-09-19T21:36:48.199825+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000243341,"stat_sum":{"num_bytes":60057909,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":12745,"num_read_kb":18492,"num_write":775,"num_write_kb":4812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":136,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.190026+0000","last_change":"2024-09-19T21:36:02.839832+0000","last_active":"2024-09-19T21:36:47.190026+0000","last_peered":"2024-09-19T21:36:47.190026+0000","last_clean":"2024-09-19T21:36:47.190026+0000","last_became_active":"2024-09-19T21:36:02.830895+0000","last_became_peered":"2024-09-19T21:36:02.830895+0000","last_unstale":"2024-09-19T21:36:47.190026+0000","last_undegraded":"2024-09-19T21:36:47.190026+0000","last_fullsized":"2024-09-19T21:36:47.190026+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:40.111547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"109'450","reported_seq":11453,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.192962+0000","last_change":"2024-09-19T21:36:38.063350+0000","last_active":"2024-09-19T21:36:47.192962+0000","last_peered":"2024-09-19T21:36:47.192962+0000","last_clean":"2024-09-19T21:36:47.192962+0000","last_became_active":"2024-09-19T21:36:38.061093+0000","last_became_peered":"2024-09-19T21:36:38.061093+0000","last_unstale":"2024-09-19T21:36:47.192962+0000","last_undegraded":"2024-09-19T21:36:47.192962+0000","last_fullsized":"2024-09-19T21:36:47.192962+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:38:17.668473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038045499999999998,"stat_sum":{"num_bytes":36375145,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":12585,"num_read_kb":49685,"num_write":437,"num_write_kb":12178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":17020731,"num_keys_recovered":94,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:43.159455+0000","last_change":"2024-09-19T21:36:02.831334+0000","last_active":"2024-09-19T21:36:43.159455+0000","last_peered":"2024-09-19T21:36:43.159455+0000","last_clean":"2024-09-19T21:36:43.159455+0000","last_became_active":"2024-09-19T21:36:02.830988+0000","last_became_peered":"2024-09-19T21:36:02.830988+0000","last_unstale":"2024-09-19T21:36:43.159455+0000","last_undegraded":"2024-09-19T21:36:43.159455+0000","last_fullsized":"2024-09-19T21:36:43.159455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:09:35.907712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"114'646","reported_seq":8162,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.206098+0000","last_change":"2024-09-19T21:36:35.058401+0000","last_active":"2024-09-19T21:36:48.206098+0000","last_peered":"2024-09-19T21:36:48.206098+0000","last_clean":"2024-09-19T21:36:48.206098+0000","last_became_active":"2024-09-19T21:36:35.056705+0000","last_became_peered":"2024-09-19T21:36:35.056705+0000","last_unstale":"2024-09-19T21:36:48.206098+0000","last_undegraded":"2024-09-19T21:36:48.206098+0000","last_fullsized":"2024-09-19T21:36:48.206098+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:15:07.237414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00179126,"stat_sum":{"num_bytes":88874676,"num_objects":77,"num_object_clones":24,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":28,"num_read":9298,"num_read_kb":32973,"num_write":911,"num_write_kb":10324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":13548106,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.192856+0000","last_change":"2024-09-19T21:36:32.033895+0000","last_active":"2024-09-19T21:36:47.192856+0000","last_peered":"2024-09-19T21:36:47.192856+0000","last_clean":"2024-09-19T21:36:47.192856+0000","last_became_active":"2024-09-19T21:36:32.033629+0000","last_became_peered":"2024-09-19T21:36:32.033629+0000","last_unstale":"2024-09-19T21:36:47.192856+0000","last_undegraded":"2024-09-19T21:36:47.192856+0000","last_fullsized":"2024-09-19T21:36:47.192856+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:52:25.528924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"110'381","reported_seq":11184,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-19T21:36:45.181587+0000","last_change":"2024-09-19T21:36:35.050437+0000","last_active":"2024-09-19T21:36:45.181587+0000","last_peered":"2024-09-19T21:36:45.181587+0000","last_clean":"2024-09-19T21:36:45.181587+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:36:45.181587+0000","last_undegraded":"2024-09-19T21:36:45.181587+0000","last_fullsized":"2024-09-19T21:36:45.181587+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078698199999999998,"stat_sum":{"num_bytes":136467521,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":12084,"num_read_kb":38943,"num_write":416,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":135,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196364+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:36:47.196364+0000","last_peered":"2024-09-19T21:36:47.196364+0000","last_clean":"2024-09-19T21:36:47.196364+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:36:47.196364+0000","last_undegraded":"2024-09-19T21:36:47.196364+0000","last_fullsized":"2024-09-19T21:36:47.196364+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":321,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196177+0000","last_change":"2024-09-19T21:36:37.060633+0000","last_active":"2024-09-19T21:36:47.196177+0000","last_peered":"2024-09-19T21:36:47.196177+0000","last_clean":"2024-09-19T21:36:47.196177+0000","last_became_active":"2024-09-19T21:36:37.060036+0000","last_became_peered":"2024-09-19T21:36:37.060036+0000","last_unstale":"2024-09-19T21:36:47.196177+0000","last_undegraded":"2024-09-19T21:36:47.196177+0000","last_fullsized":"2024-09-19T21:36:47.196177+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:41:47.787667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"114'434","reported_seq":15180,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.190149+0000","last_change":"2024-09-19T21:36:37.060544+0000","last_active":"2024-09-19T21:36:48.190149+0000","last_peered":"2024-09-19T21:36:48.190149+0000","last_clean":"2024-09-19T21:36:48.190149+0000","last_became_active":"2024-09-19T21:36:37.059116+0000","last_became_peered":"2024-09-19T21:36:37.059116+0000","last_unstale":"2024-09-19T21:36:48.190149+0000","last_undegraded":"2024-09-19T21:36:48.190149+0000","last_fullsized":"2024-09-19T21:36:48.190149+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:14:10.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023849079999999998,"stat_sum":{"num_bytes":113242200,"num_objects":88,"num_object_clones":26,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":32,"num_read":14193,"num_read_kb":61228,"num_write":514,"num_write_kb":12983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":17942192,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"114'374","reported_seq":5417,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199547+0000","last_change":"2024-09-19T21:34:32.006176+0000","last_active":"2024-09-19T21:36:48.199547+0000","last_peered":"2024-09-19T21:36:48.199547+0000","last_clean":"2024-09-19T21:36:48.199547+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:36:48.199547+0000","last_undegraded":"2024-09-19T21:36:48.199547+0000","last_fullsized":"2024-09-19T21:36:48.199547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045413400000000003,"stat_sum":{"num_bytes":103381759,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":21,"num_read":5804,"num_read_kb":18673,"num_write":350,"num_write_kb":6828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"114'487","reported_seq":7830,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199602+0000","last_change":"2024-09-19T21:34:32.006420+0000","last_active":"2024-09-19T21:36:48.199602+0000","last_peered":"2024-09-19T21:36:48.199602+0000","last_clean":"2024-09-19T21:36:48.199602+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:36:48.199602+0000","last_undegraded":"2024-09-19T21:36:48.199602+0000","last_fullsized":"2024-09-19T21:36:48.199602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000473516,"stat_sum":{"num_bytes":44002612,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":27,"num_read":9234,"num_read_kb":47108,"num_write":584,"num_write_kb":13272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"111'255","reported_seq":2109,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-19T21:36:46.193027+0000","last_change":"2024-09-19T21:34:32.005781+0000","last_active":"2024-09-19T21:36:46.193027+0000","last_peered":"2024-09-19T21:36:46.193027+0000","last_clean":"2024-09-19T21:36:46.193027+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:36:46.193027+0000","last_undegraded":"2024-09-19T21:36:46.193027+0000","last_fullsized":"2024-09-19T21:36:46.193027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010677900000000001,"stat_sum":{"num_bytes":22356612,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":13,"num_read":2111,"num_read_kb":20155,"num_write":281,"num_write_kb":6539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"110'191","reported_seq":2988,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.199892+0000","last_change":"2024-09-19T21:34:32.006522+0000","last_active":"2024-09-19T21:36:48.199892+0000","last_peered":"2024-09-19T21:36:48.199892+0000","last_clean":"2024-09-19T21:36:48.199892+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:36:48.199892+0000","last_undegraded":"2024-09-19T21:36:48.199892+0000","last_fullsized":"2024-09-19T21:36:48.199892+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064455700000000003,"stat_sum":{"num_bytes":17011712,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":2522,"num_read_kb":12510,"num_write":180,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"114'396","reported_seq":9273,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.224054+0000","last_change":"2024-09-19T21:34:32.005110+0000","last_active":"2024-09-19T21:36:48.224054+0000","last_peered":"2024-09-19T21:36:48.224054+0000","last_clean":"2024-09-19T21:36:48.224054+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:36:48.224054+0000","last_undegraded":"2024-09-19T21:36:48.224054+0000","last_fullsized":"2024-09-19T21:36:48.224054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029737129999999998,"stat_sum":{"num_bytes":122207525,"num_objects":86,"num_object_clones":27,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":31,"num_read":9296,"num_read_kb":30953,"num_write":435,"num_write_kb":9593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"110'496","reported_seq":5988,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.583211+0000","last_change":"2024-09-19T21:36:36.059649+0000","last_active":"2024-09-19T21:36:47.583211+0000","last_peered":"2024-09-19T21:36:47.583211+0000","last_clean":"2024-09-19T21:36:47.583211+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:36:47.583211+0000","last_undegraded":"2024-09-19T21:36:47.583211+0000","last_fullsized":"2024-09-19T21:36:47.583211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035219500000000003,"stat_sum":{"num_bytes":33208372,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":8078,"num_read_kb":50986,"num_write":422,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"114'715","reported_seq":4133,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.230389+0000","last_change":"2024-09-19T21:34:32.006768+0000","last_active":"2024-09-19T21:36:48.230389+0000","last_peered":"2024-09-19T21:36:48.230389+0000","last_clean":"2024-09-19T21:36:48.230389+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:36:48.230389+0000","last_undegraded":"2024-09-19T21:36:48.230389+0000","last_fullsized":"2024-09-19T21:36:48.230389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048102000000000003,"stat_sum":{"num_bytes":11077999,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":6760,"num_read_kb":10636,"num_write":801,"num_write_kb":2930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"114'484","reported_seq":7890,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.221884+0000","last_change":"2024-09-19T21:36:05.849347+0000","last_active":"2024-09-19T21:36:48.221884+0000","last_peered":"2024-09-19T21:36:48.221884+0000","last_clean":"2024-09-19T21:36:48.221884+0000","last_became_active":"2024-09-19T21:36:05.848208+0000","last_became_peered":"2024-09-19T21:36:05.848208+0000","last_unstale":"2024-09-19T21:36:48.221884+0000","last_undegraded":"2024-09-19T21:36:48.221884+0000","last_fullsized":"2024-09-19T21:36:48.221884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:25:57.433432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064145399999999996,"stat_sum":{"num_bytes":47277705,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":35,"num_read":9577,"num_read_kb":56968,"num_write":438,"num_write_kb":16073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":13819481,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"110'311","reported_seq":3534,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.708462+0000","last_change":"2024-09-19T21:36:35.057415+0000","last_active":"2024-09-19T21:36:47.708462+0000","last_peered":"2024-09-19T21:36:47.708462+0000","last_clean":"2024-09-19T21:36:47.708462+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:36:47.708462+0000","last_undegraded":"2024-09-19T21:36:47.708462+0000","last_fullsized":"2024-09-19T21:36:47.708462+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046798599999999999,"stat_sum":{"num_bytes":80246308,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":3753,"num_read_kb":19636,"num_write":236,"num_write_kb":2823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"113'229","reported_seq":6818,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-19T21:36:48.228478+0000","last_change":"2024-09-19T21:36:05.847174+0000","last_active":"2024-09-19T21:36:48.228478+0000","last_peered":"2024-09-19T21:36:48.228478+0000","last_clean":"2024-09-19T21:36:48.228478+0000","last_became_active":"2024-09-19T21:36:05.846322+0000","last_became_peered":"2024-09-19T21:36:05.846322+0000","last_unstale":"2024-09-19T21:36:48.228478+0000","last_undegraded":"2024-09-19T21:36:48.228478+0000","last_fullsized":"2024-09-19T21:36:48.228478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:34.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000347931,"stat_sum":{"num_bytes":84715008,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":6453,"num_read_kb":69979,"num_write":243,"num_write_kb":8535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'327","reported_seq":7492,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-19T21:36:47.196241+0000","last_change":"2024-09-19T21:34:32.000935+0000","last_active":"2024-09-19T21:36:47.196241+0000","last_peered":"2024-09-19T21:36:47.196241+0000","last_clean":"2024-09-19T21:36:47.196241+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:36:47.196241+0000","last_undegraded":"2024-09-19T21:36:47.196241+0000","last_fullsized":"2024-09-19T21:36:47.196241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063325299999999996,"stat_sum":{"num_bytes":34933412,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":7953,"num_read_kb":20587,"num_write":347,"num_write_kb":5705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2021284822,"num_objects":1995,"num_object_clones":545,"num_object_copies":3990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":730,"num_read":285046,"num_read_kb":1018320,"num_write":15967,"num_write_kb":259132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":173104204,"num_keys_recovered":458,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446402560,"data_stored":590238924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14564,"ondisk_log_size":14564,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95588,"kb_used_data":68568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94276252,"statfs":{"total":96636764160,"available":96538882048,"internally_reserved":0,"allocated":70213632,"data_stored":90760278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88100000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0640000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.41499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.004,"5min":1.048,"15min":1.048},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.98399999999999999,"5min":0.996,"15min":0.996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.024,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.25,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2889999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123940,"kb_used_data":66580,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94247900,"statfs":{"total":96636764160,"available":96509849600,"internally_reserved":0,"allocated":68177920,"data_stored":81544465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.872},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4330000000000001,"5min":1.921,"15min":1.921},"last":0.84499999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.879,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.626,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.81699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133316,"kb_used_data":80052,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94238524,"statfs":{"total":96636764160,"available":96500248576,"internally_reserved":0,"allocated":81973248,"data_stored":112216372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":2.5169999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":3.5750000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.66700000000000004,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":3.5379999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":3.6059999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.625}]},{"osd":5,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.266,"5min":1.282,"15min":1.282},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.86599999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":3.637}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145988,"kb_used_data":84516,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94225852,"statfs":{"total":96636764160,"available":96487272448,"internally_reserved":0,"allocated":86544384,"data_stored":122423493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.96099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.87,"5min":1.333,"15min":1.333},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.012,"15min":1.012},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.63},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0900000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0740000000000001,"5min":1.718,"15min":1.718},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.93500000000000005}]}]},{"osd":3,"up_from":12,"seq":51539607586,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142152,"kb_used_data":88888,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94229688,"statfs":{"total":96636764160,"available":96491200512,"internally_reserved":0,"allocated":91021312,"data_stored":112543422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92000000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0269999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":1.175,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":3.2719999999999998,"5min":1.2330000000000001,"15min":0.89300000000000002},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":42.182000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.78400000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.99399999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.105,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":85417984,"data_stored":122104692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80248832,"data_stored":111279848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66715648,"data_stored":80870173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":89706496,"data_stored":112005716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69013504,"data_stored":90348539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":55300096,"data_stored":73629956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:36:51.944 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-19T21:36:51.944 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:51.944 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T21:36:51.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:36:51.982 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:51.978+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:52.046 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-19T21:36:52.048 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.048 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:52.066 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:52.058+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:52.167 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:52.163+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:52.175 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.175 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:36:52.267 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:52.262+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:52.321 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.321 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 118, 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:36:52.360 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:36:52.232544+0000", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:36:31.002760+0000", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5799936, 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:36:52.361 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "luminous", 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.362 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:52.363 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:52.364 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:52.365 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:52.366 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:52.367 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:36:52.368 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:52.369 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:36:52.370 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:52.371 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:52.372 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:52.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:52.375 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.2699999809265137, 2024-09-19T21:36:52.376 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.2699999809265137, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:52.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:36:52.379 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "117", 2024-09-19T21:36:52.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 44, 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 117, 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:52.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:52.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:52.383 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.5099999904632568, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.5099999904632568, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.25, 2024-09-19T21:36:52.384 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.25, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:52.385 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:36:52.387 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.389 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.390 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:52.391 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:36:52.392 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.395 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.396 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:52.397 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 107, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:36:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:52.403 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 105, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.405 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:36:52.409 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:36:52.410 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.412 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.413 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:36:52.414 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 0, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:52.415 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:52.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.420 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:36:52.421 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.422 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:36:52.423 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:36:52.424 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:52.425 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 65536, 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [ 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "pgid": "2.2", 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: 5, 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: 0 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:52.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:36:52.427 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:36:52.428 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:52.429 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:52.432 INFO:tasks.thrashosds.thrasher:j is {'epoch': 118, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:36:52.232544+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:36:31.002760+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.2', 'osds': [5, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:36:52.432 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.2 2024-09-19T21:36:52.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.2 2024-09-19T21:36:52.433 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:52.427+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:52.531 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:52.527+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:52.547 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.547 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.557 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:52.565 INFO:teuthology.orchestra.run.smithi122.stdout:5905 read 0x760d600 thru 0x760e8b4 (0x12b5 bytes) 2024-09-19T21:36:52.566 INFO:teuthology.orchestra.run.smithi122.stdout:5906 write 0x826e200 thru 0x826f328 (0x1129 bytes) 2024-09-19T21:36:52.567 INFO:teuthology.orchestra.run.smithi122.stdout:5907 write 0x8df3800 thru 0x8e0214f (0xe950 bytes) 2024-09-19T21:36:52.568 INFO:teuthology.orchestra.run.smithi122.stdout:5910 read 0x3ee4000 thru 0x3eed7c5 (0x97c6 bytes) 2024-09-19T21:36:52.573 INFO:teuthology.orchestra.run.smithi122.stdout:5911 trunc from 0xc3f4200 to 0xd144c00 2024-09-19T21:36:52.608 INFO:teuthology.orchestra.run.smithi122.stdout:5912 read 0x98d6800 thru 0x98da1c6 (0x39c7 bytes) 2024-09-19T21:36:52.630 INFO:teuthology.orchestra.run.smithi122.stdout:5914 trunc from 0xd144c00 to 0x9978600 2024-09-19T21:36:52.632 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:52.627+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:52.659 INFO:teuthology.orchestra.run.smithi122.stdout:5915 read 0x8f6ce00 thru 0x8f79f0b (0xd10c bytes) 2024-09-19T21:36:52.671 INFO:teuthology.orchestra.run.smithi122.stdout:5916 trunc from 0x9978600 to 0xd46c800 2024-09-19T21:36:52.690 INFO:teuthology.orchestra.run.smithi122.stdout:5917 write 0x358d400 thru 0x359450c (0x710d bytes) 2024-09-19T21:36:52.693 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.693 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.693 INFO:teuthology.orchestra.run.smithi122.stdout:5918 read 0x69ea00 thru 0x6a8ef7 (0xa4f8 bytes) 2024-09-19T21:36:52.693 INFO:teuthology.orchestra.run.smithi122.stdout:5919 write 0xb2a1400 thru 0xb2a53a3 (0x3fa4 bytes) 2024-09-19T21:36:52.694 INFO:teuthology.orchestra.run.smithi122.stdout:5921 read 0x4381400 thru 0x4385358 (0x3f59 bytes) 2024-09-19T21:36:52.694 INFO:teuthology.orchestra.run.smithi122.stdout:5923 write 0x65ace00 thru 0x65ba8bd (0xdabe bytes) 2024-09-19T21:36:52.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:52.750 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:52.746+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:52.751 INFO:teuthology.orchestra.run.smithi122.stdout:5926 write 0xe9c7200 thru 0xe9c93c1 (0x21c2 bytes) 2024-09-19T21:36:52.754 INFO:teuthology.orchestra.run.smithi122.stdout:5927 punch from 0xbee8e00 to 0xbeec000, (0x3200 bytes) 2024-09-19T21:36:52.754 INFO:teuthology.orchestra.run.smithi122.stdout:5930 write 0x5e4ac00 thru 0x5e525f6 (0x79f7 bytes) 2024-09-19T21:36:52.754 INFO:teuthology.orchestra.run.smithi122.stdout:5931 read 0xa30ac00 thru 0xa312144 (0x7545 bytes) 2024-09-19T21:36:52.754 INFO:teuthology.orchestra.run.smithi122.stdout:5932 write 0xbd4bc00 thru 0xbd54e0b (0x920c bytes) 2024-09-19T21:36:52.754 INFO:teuthology.orchestra.run.smithi122.stdout:5933 punch from 0x626d400 to 0x627a800, (0xd400 bytes) 2024-09-19T21:36:52.757 INFO:teuthology.orchestra.run.smithi122.stdout:5934 punch from 0xe87e400 to 0xe884600, (0x6200 bytes) 2024-09-19T21:36:52.803 INFO:teuthology.orchestra.run.smithi122.stdout:5935 trunc from 0xe9c9400 to 0x4515a00 2024-09-19T21:36:52.833 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.833 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.833 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:52.831+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:52.844 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:52.865 INFO:teuthology.orchestra.run.smithi122.stdout:5936 trunc from 0x4515a00 to 0x1935a00 2024-09-19T21:36:52.915 INFO:teuthology.orchestra.run.smithi122.stdout:5937 trunc from 0x1935a00 to 0x5295200 2024-09-19T21:36:52.934 INFO:teuthology.orchestra.run.smithi122.stdout:5939 punch from 0x1512400 to 0x1515000, (0x2c00 bytes) 2024-09-19T21:36:52.934 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:52.930+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:52.959 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:52.959 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:52.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:52.974 INFO:teuthology.orchestra.run.smithi122.stdout:5940 write 0xdcd1e00 thru 0xdcd7f7f (0x6180 bytes) 2024-09-19T21:36:52.976 INFO:teuthology.orchestra.run.smithi122.stdout:5941 read 0xd7c7e00 thru 0xd7cbe77 (0x4078 bytes) 2024-09-19T21:36:52.976 INFO:teuthology.orchestra.run.smithi122.stdout:5944 punch from 0xad8a800 to 0xad8da00, (0x3200 bytes) 2024-09-19T21:36:52.976 INFO:teuthology.orchestra.run.smithi122.stdout:5946 read 0x8d38c00 thru 0x8d43d1c (0xb11d bytes) 2024-09-19T21:36:52.986 INFO:teuthology.orchestra.run.smithi122.stdout:5947 trunc from 0xdcd8000 to 0x9a61000 2024-09-19T21:36:53.012 INFO:teuthology.orchestra.run.smithi122.stdout:5949 read 0x4b6fa00 thru 0x4b7e38f (0xe990 bytes) 2024-09-19T21:36:53.014 INFO:teuthology.orchestra.run.smithi122.stdout:5951 punch from 0x5285a00 to 0x5293800, (0xde00 bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5952 read 0x2537800 thru 0x254544b (0xdc4c bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5953 read 0x2d3a800 thru 0x2d3e800 (0x4001 bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5955 write 0x3848400 thru 0x384e3fb (0x5ffc bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5956 read 0x21b0800 thru 0x21b701c (0x681d bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5957 read 0x32dc800 thru 0x32e3cb7 (0x74b8 bytes) 2024-09-19T21:36:53.016 INFO:teuthology.orchestra.run.smithi122.stdout:5958 punch from 0x8be4600 to 0x8bf3800, (0xf200 bytes) 2024-09-19T21:36:53.017 INFO:teuthology.orchestra.run.smithi122.stdout:5959 read 0x1eae600 thru 0x1eaec59 (0x65a bytes) 2024-09-19T21:36:53.018 INFO:teuthology.orchestra.run.smithi122.stdout:5961 write 0x13db200 thru 0x13db8cc (0x6cd bytes) 2024-09-19T21:36:53.018 INFO:teuthology.orchestra.run.smithi122.stdout:5962 punch from 0x12e9e00 to 0x12f3000, (0x9200 bytes) 2024-09-19T21:36:53.020 INFO:teuthology.orchestra.run.smithi122.stdout:5963 read 0x7f27c00 thru 0x7f34f74 (0xd375 bytes) 2024-09-19T21:36:53.021 INFO:teuthology.orchestra.run.smithi122.stdout:5965 write 0x1c5d800 thru 0x1c60399 (0x2b9a bytes) 2024-09-19T21:36:53.035 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:53.031+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:53.048 INFO:teuthology.orchestra.run.smithi122.stdout:5966 trunc from 0x9a61000 to 0x3d8600 2024-09-19T21:36:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.132 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.136 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:53.131+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:53.138 INFO:teuthology.orchestra.run.smithi122.stdout:5968 trunc from 0x3d8600 to 0xc829c00 2024-09-19T21:36:53.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:53.161 INFO:teuthology.orchestra.run.smithi122.stdout:5969 trunc from 0xc829c00 to 0xa365800 2024-09-19T21:36:53.179 INFO:teuthology.orchestra.run.smithi122.stdout:5970 punch from 0x7118000 to 0x711d400, (0x5400 bytes) 2024-09-19T21:36:53.191 INFO:teuthology.orchestra.run.smithi122.stdout:5971 trunc from 0xa365800 to 0x5a9a200 2024-09-19T21:36:53.217 INFO:teuthology.orchestra.run.smithi122.stdout:5974 read 0x41be600 thru 0x41caf40 (0xc941 bytes) 2024-09-19T21:36:53.220 INFO:teuthology.orchestra.run.smithi122.stdout:5975 write 0x3743e00 thru 0x37467b7 (0x29b8 bytes) 2024-09-19T21:36:53.220 INFO:teuthology.orchestra.run.smithi122.stdout:5976 write 0x164ce00 thru 0x1650a5d (0x3c5e bytes) 2024-09-19T21:36:53.224 INFO:teuthology.orchestra.run.smithi122.stdout:5977 trunc from 0x5a9a200 to 0x73db600 2024-09-19T21:36:53.237 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:53.235+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:53.255 INFO:teuthology.orchestra.run.smithi012.stderr:clear 2.2 pg_upmap mapping 2024-09-19T21:36:53.260 INFO:teuthology.orchestra.run.smithi122.stdout:5978 read 0xd48200 thru 0xd4cc22 (0x4a23 bytes) 2024-09-19T21:36:53.260 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.260 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:53.279 INFO:teuthology.orchestra.run.smithi122.stdout:5980 write 0x7762000 thru 0x776b6a1 (0x96a2 bytes) 2024-09-19T21:36:53.282 INFO:teuthology.orchestra.run.smithi122.stdout:5981 punch from 0x4a56400 to 0x4a61a00, (0xb600 bytes) 2024-09-19T21:36:53.283 INFO:teuthology.orchestra.run.smithi122.stdout:5984 read 0x71bb600 thru 0x71c14a5 (0x5ea6 bytes) 2024-09-19T21:36:53.311 INFO:teuthology.orchestra.run.smithi122.stdout:5985 write 0xc5de200 thru 0xc5df83a (0x163b bytes) 2024-09-19T21:36:53.312 INFO:teuthology.orchestra.run.smithi122.stdout:5986 read 0xb21ea00 thru 0xb2291cb (0xa7cc bytes) 2024-09-19T21:36:53.313 INFO:teuthology.orchestra.run.smithi122.stdout:5987 read 0x3de800 thru 0x3e736f (0x8b70 bytes) 2024-09-19T21:36:53.316 INFO:teuthology.orchestra.run.smithi122.stdout:5988 flatten 2024-09-19T21:36:53.337 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:53.335+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:53.339 INFO:teuthology.orchestra.run.smithi122.stdout:5990 read 0x2990200 thru 0x299f303 (0xf104 bytes) 2024-09-19T21:36:53.340 INFO:teuthology.orchestra.run.smithi122.stdout:5991 punch from 0x8a0c800 to 0x8a0d200, (0xa00 bytes) 2024-09-19T21:36:53.341 INFO:teuthology.orchestra.run.smithi122.stdout:5992 write 0x270c800 thru 0x270d850 (0x1051 bytes) 2024-09-19T21:36:53.341 INFO:teuthology.orchestra.run.smithi122.stdout:5993 read 0x68cbc00 thru 0x68d2cb2 (0x70b3 bytes) 2024-09-19T21:36:53.341 INFO:teuthology.orchestra.run.smithi122.stdout:5995 punch from 0x651200 to 0x654a00, (0x3800 bytes) 2024-09-19T21:36:53.341 INFO:teuthology.orchestra.run.smithi122.stdout:5996 punch from 0xc527800 to 0xc52d200, (0x5a00 bytes) 2024-09-19T21:36:53.342 INFO:teuthology.orchestra.run.smithi122.stdout:5997 read 0x4dfa00 thru 0x4e6122 (0x6723 bytes) 2024-09-19T21:36:53.344 INFO:teuthology.orchestra.run.smithi122.stdout:5998 read 0x7ac000 thru 0x7b9f70 (0xdf71 bytes) 2024-09-19T21:36:53.346 INFO:teuthology.orchestra.run.smithi122.stdout:5999 write 0x2c4f800 thru 0x2c516be (0x1ebf bytes) 2024-09-19T21:36:53.346 INFO:teuthology.orchestra.run.smithi122.stdout:6000 write 0x5236c00 thru 0x52419b1 (0xadb2 bytes) 2024-09-19T21:36:53.391 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.391 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:53.438 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:53.434+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:53.516 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.516 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:53.539 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:53.538+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:53.635 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.636 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:53.639 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:53.635+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:53.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:53.664 INFO:teuthology.orchestra.run.smithi122.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-19T21:36:53.740 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:53.735+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:53.796 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:53.817 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:53.817 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:53.821 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:53.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:53.841 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:53.834+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:53.843 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:53.865 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:53.887 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:53.911 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:53.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:53.914 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:53.941 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:53.934+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:53.946 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:53.946 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:53.946 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:53.946 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:53.946 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:53.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:54.043 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:54.038+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.868632039999994, 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:54.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.868052129000006, 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.076 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867989481999999, 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867923161999997, 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:54.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867219484000003, 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:54.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867146661999996, 2024-09-19T21:36:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:54.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867132924000003, 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:54.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.102 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867115235, 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:54.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867064849000002, 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:54.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.867037960000005, 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.112 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:54.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.866997384000001, 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:54.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.120 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.866975769999996, 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 85.866956571000003, 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.125 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:54.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.377638969000003, 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.377015486999994, 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:54.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:54.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.139 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 72.376683211, 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:54.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:54.144 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.287591994, 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:36:54.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:36:54.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.264995638999999, 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:54.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:36:54.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.264664971999998, 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:36:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.160 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 17.2641454, 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:36:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.165 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:54.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:54.167 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:54.163+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:54.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:36:54.267 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:54.263+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:54.284 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:54.284 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:54.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:54.368 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:54.362+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:54.397 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:54.397 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:54.397 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:54.397 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:54.397 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:54.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:54.469 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:54.467+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:54.513 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 118.760671413, 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:54.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133332017000001, 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:54.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:54.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133279551000001, 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:54.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.531 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133249004000007, 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:54.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:54.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133226406999995, 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:54.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:54.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:54.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133203954999999, 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:36:54.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133189658999996, 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:36:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:36:54.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133173032000002, 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:36:54.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133154395999995, 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:36:54.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:36:54.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:36:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133127731000002, 2024-09-19T21:36:54.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:36:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.568 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133117519999999, 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:36:54.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:36:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:36:54.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.133105033999996, 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:36:54.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129230444000001, 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:36:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:36:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129212925000004, 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:54.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129175371000002, 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.589 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:36:54.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:36:54.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129157527999993, 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:54.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:36:54.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129145078999997, 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129134386999993, 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.606 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 74.129122809999998, 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:36:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 73.713150854000006, 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:36:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:54.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:54.616 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:54.616 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:54.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:54.618 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:54.614+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:54.718 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:54.714+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:54.725 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:54.725 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:54.725 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:54.734 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:54.819 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:54.814+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:54.846 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:54.846 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:54.846 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:54.846 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:54.846 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:54.856 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:54.920 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:54.914+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.344418536000006, 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:36:54.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:36:54.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:54.973 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.344396807999999, 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:36:54.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.344376076000003, 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:36:54.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.982 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.344341639999996, 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:36:54.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:36:54.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.344298843000004, 2024-09-19T21:36:54.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:36:54.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.991 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.339073333000002, 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:36:54.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:54.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:54.996 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.339051034999997, 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:36:54.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:36:54.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.339020180000006, 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:36:55.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:36:55.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338977471000007, 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.005 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:36:55.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:36:55.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338953484000001, 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:36:55.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.013 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338922080000003, 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:36:55.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338896914000003, 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:36:55.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.022 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338851836000003, 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:36:55.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338825091999993, 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:36:55.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.028 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:55.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:36:55.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:36:55.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:36:55.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:36:55.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:36:55.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:36:55.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.035 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338811800000002, 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:36:55.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.037 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:36:55.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:36:55.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.338794880999998, 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.041 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.045 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.938808109, 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:55.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:36:55.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:36:55.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:55.051 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.937241405999998, 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:36:55.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:55.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:36:55.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.937205125999999, 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:55.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:55.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:36:55.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.066 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.937185104000001, 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:36:55.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:36:55.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:36:55.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:36:55.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:36:55.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:36:55.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.074 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:55.076 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:55.070+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:55.177 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:55.170+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:55.236 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:55.236 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:55.236 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:55.271 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:36:55.278 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:55.275+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:55.371 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:55.371 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:55.371 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:55.371 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:55.371 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.378 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:55.374+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:55.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:55.479 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:55.475+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:55.498 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:55.498 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 159.56301041, 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:36:55.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:36:55.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:55.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:36:55.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.505 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243443079000002, 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:36:55.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:36:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:36:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243431313000002, 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:36:55.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:36:55.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243418419999998, 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:36:55.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:36:55.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:55.522 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243406397000001, 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:36:55.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243388753000005, 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:36:55.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243372957000005, 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.531 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:55.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:36:55.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:36:55.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243361105999995, 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:36:55.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.540 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243292261999997, 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:36:55.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:36:55.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:36:55.545 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243264121999999, 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.243240993000001, 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.551 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:36:55.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.228464372999994, 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.555 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:55.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:36:55.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.221128058000005, 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.560 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:36:55.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.564 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.221107168000003, 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:36:55.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.214031628000001, 2024-09-19T21:36:55.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:36:55.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.214011084000006, 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.033657012999996, 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.578 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:36:55.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.033646586000003, 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.583 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:36:55.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:36:55.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.033636157000004, 2024-09-19T21:36:55.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:36:55.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 83.033626093999999, 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.592 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:55.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:36:55.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:36:55.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:55.596 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:55.598 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:55.595+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:55.698 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:55.695+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:55.709 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:55.709 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:55.709 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:36:55.709 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.718 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:55.799 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:55.794+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:55.822 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:55.822 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:36:55.822 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:36:55.822 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:36:55.822 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:55.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:55.899 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:55.894+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:55.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:36:55.999 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:55.995+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:56.015 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.2846605, 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:56.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.021 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.682705256000006, 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.682259274000003, 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:56.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.032 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.682145094999996, 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:56.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:56.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:56.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.682128661999997, 2024-09-19T21:36:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:56.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.682113106000003, 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:56.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:56.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681999036999997, 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.051 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681982074000004, 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681966090000003, 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:56.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:56.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681948915000007, 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:56.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.065 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681931716999998, 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681914886000001, 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:56.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:56.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681898364999995, 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:56.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681875761000001, 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:56.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681859372999995, 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.084 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:56.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:56.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.681750492000006, 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 25.002719457000001, 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 25.002606308000001, 2024-09-19T21:36:56.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:56.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:56.102 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 24.993340795000002, 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:56.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:56.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:56.107 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 24.993268369999999, 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:56.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:56.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:56.112 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:56.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:56.114 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:56.115 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:56.115 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:56.111+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:56.116 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:56.139 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:56.163 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:56.186 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:56.214 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:56.210+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:56.235 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.235 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:56.315 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:56.311+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:56.360 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.360 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:56.416 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:56.410+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:56.477 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.478 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:56.517 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:56.515+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:56.618 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:56.614+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:56.719 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:56.714+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:56.749 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.750 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:56.820 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:56.814+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:56.869 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:56.869 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:56.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:56.921 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:56.918+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:57.022 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:57.018+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:57.031 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.031 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:57.123 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:57.118+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:57.147 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.148 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:57.224 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:57.218+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:57.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:36:57.318 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.318 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:36:57.334 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:57.330+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:57.425 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:57.422+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:57.452 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.452 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:36:57.525 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:57.518+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:57.566 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.567 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:36:57.626 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:57.622+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:57.683 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.683 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:36:57.726 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:57.722+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:57.800 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.801 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.810 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:36:57.827 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:57.822+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:57.916 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:57.917 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:57.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:36:57.970 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:57.966+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:58.029 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:58.026+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:58.081 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:58.081 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:58.092 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:36:58.129 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:58.122+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:58.196 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:58.196 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:58.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:36:58.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:36:58.246 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:58.242+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:58.268 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:36:58.268 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:36:58.268 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-19T21:36:58.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:36:58.304 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:36:58.331 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:58.326+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:58.355 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:36:58.370 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:36:58.370 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:36:58.375 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:36:58.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:36:58.396 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:36:58.432 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:36:58.433 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:58.426+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:58.463 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:36:58.485 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:36:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:36:58.506 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 122.776005813, 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:58.507 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:36:58.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:36:58.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.513 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.174050569000002, 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:36:58.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:36:58.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173604587, 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:58.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173490408000006, 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:36:58.526 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173473974999993, 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:36:58.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173458418999999, 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:58.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.535 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173344349999994, 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:58.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173327387, 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:36:58.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173311403, 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:36:58.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173294228000003, 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.549 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:36:58.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173277029999994, 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:36:58.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:36:58.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.558 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173260198999998, 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.563 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173243678000006, 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:36:58.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.568 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173221073999997, 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:36:58.572 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173204686000005, 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.576 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.173095805000003, 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.577 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:36:58.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.580 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.494064770000001, 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:36:58.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:36:58.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.493951621000001, 2024-09-19T21:36:58.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:58.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.484686107999998, 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:36:58.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:36:58.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:36:58.596 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.484613682999999, 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:58.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.601 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:58.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:36:58.603 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:58.603 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:36:58.603 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 123, 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:36:58.275832+0000", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:36:31.002760+0000", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5806080, 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "nodeep-scrub", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "noscrub", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:36:58.604 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "jewel", 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:36:58.605 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:58.606 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:58.607 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:58.608 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:58.609 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:58.610 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:36:58.611 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:36:58.612 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:58.613 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:36:58.614 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:58.615 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:58.616 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:58.617 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.2699999809265137, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.2699999809265137, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:36:58.618 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:36:58.619 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "123", 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:36:58.620 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 48, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 123, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:36:58.621 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:36:58.622 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:36:58.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.5099999904632568, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.5099999904632568, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.25, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.25, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.627 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.628 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.629 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.630 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.631 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.632 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 107, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.636 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.637 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.638 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:36:58.640 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.641 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 105, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:36:58.642 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:36:58.643 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.644 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.645 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 106, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.647 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.648 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.650 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.651 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:36:58.652 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 23, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:36:58.653 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.654 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:36:58.658 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.660 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.661 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:36:58.662 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 65536, 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.663 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:36:58.664 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [ 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "snaps": [ 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 41, 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 4 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [ 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.665 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "snaps": [ 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 41, 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 1 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:36:58.666 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:36:58.667 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:36:58.667 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:36:58.667 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:36:58.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:36:58.667 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:36:58.667 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:58.598+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:58.669 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:36:58.275832+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:36:31.002760+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 4}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 41, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:36:58.669 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:36:58.713 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:36:58.710+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:36:58.783 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:58.783 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:58.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:36:58.803 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:36:58.798+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:36:58.904 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:58.898+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:58.905 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:58.905 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:58.905 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:58.905 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:58.905 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:58.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:36:59.005 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:58.998+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:59.021 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763823053, 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.022 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:36:59.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:36:59.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763809514, 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:36:59.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763783774, 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76376956, 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.040 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763748643, 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:36:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:36:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76373326700001, 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:36:59.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:36:59.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76371833899999, 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:36:59.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763665719, 2024-09-19T21:36:59.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.058 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763616141, 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:36:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.063 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763598926, 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:59.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:36:59.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.763579788, 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:36:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76356493999999, 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:36:59.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:36:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.077 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76354194699999, 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:36:59.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.082 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.762809655, 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:36:59.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76275934500001, 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:36:59.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:36:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76269703299999, 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:36:59.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:36:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.7613664, 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.100 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.761230822, 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:36:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.105 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76115406300001, 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.76109533899999, 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.110 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:36:59.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.114 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:36:59.118 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:59.110+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:59.118 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:36:59.119 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:36:59.120 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-19T21:36:58.842990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1987848527,"num_objects":1980,"num_object_clones":545,"num_object_copies":3960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1980,"num_whiteouts":730,"num_read":294116,"num_read_kb":1051869,"num_write":16765,"num_write_kb":271125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2820,"num_bytes_recovered":174875346,"num_keys_recovered":458,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15285,"ondisk_log_size":15285,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":627588,"kb_used_data":387508,"kb_used_omap":7,"kb_used_meta":239992,"kb_avail":471231612,"statfs":{"total":483183820800,"available":482541170688,"internally_reserved":0,"allocated":396808192,"data_stored":518118477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":245752032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43341367,"num_objects":-39,"num_object_clones":0,"num_object_copies":-78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-39,"num_whiteouts":0,"num_read":2290,"num_read_kb":8343,"num_write":245,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001197"},"pg_stats":[{"pgid":"3.1f","version":"122'440","reported_seq":4424,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.443103+0000","last_change":"2024-09-19T21:36:57.290273+0000","last_active":"2024-09-19T21:36:57.443103+0000","last_peered":"2024-09-19T21:36:57.443103+0000","last_clean":"2024-09-19T21:36:57.443103+0000","last_became_active":"2024-09-19T21:36:06.848734+0000","last_became_peered":"2024-09-19T21:36:06.848734+0000","last_unstale":"2024-09-19T21:36:57.443103+0000","last_undegraded":"2024-09-19T21:36:57.443103+0000","last_fullsized":"2024-09-19T21:36:57.443103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:12.838002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001594142,"stat_sum":{"num_bytes":44505733,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":17,"num_read":6339,"num_read_kb":38443,"num_write":410,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":24095839,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1e","version":"118'235","reported_seq":2409,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.272951+0000","last_change":"2024-09-19T21:36:56.273196+0000","last_active":"2024-09-19T21:36:56.272951+0000","last_peered":"2024-09-19T21:36:56.272951+0000","last_clean":"2024-09-19T21:36:56.272951+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:36:56.272951+0000","last_undegraded":"2024-09-19T21:36:56.272951+0000","last_fullsized":"2024-09-19T21:36:56.272951+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019571359999999999,"stat_sum":{"num_bytes":39146564,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":26,"num_read":1776,"num_read_kb":7796,"num_write":186,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"2"}]},{"pgid":"3.1d","version":"120'1419","reported_seq":9977,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.282636+0000","last_change":"2024-09-19T21:36:57.282877+0000","last_active":"2024-09-19T21:36:57.282636+0000","last_peered":"2024-09-19T21:36:57.282636+0000","last_clean":"2024-09-19T21:36:57.282636+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:36:57.282636+0000","last_undegraded":"2024-09-19T21:36:57.282636+0000","last_fullsized":"2024-09-19T21:36:57.282636+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098198300000000007,"stat_sum":{"num_bytes":62626163,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":16779,"num_read_kb":38929,"num_write":1859,"num_write_kb":15179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1c","version":"122'542","reported_seq":16136,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.389577+0000","last_change":"2024-09-19T21:36:57.288379+0000","last_active":"2024-09-19T21:36:57.389577+0000","last_peered":"2024-09-19T21:36:57.389577+0000","last_clean":"2024-09-19T21:36:57.389577+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:36:57.389577+0000","last_undegraded":"2024-09-19T21:36:57.389577+0000","last_fullsized":"2024-09-19T21:36:57.389577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063478799999999995,"stat_sum":{"num_bytes":43957300,"num_objects":53,"num_object_clones":17,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":17334,"num_read_kb":26468,"num_write":542,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.1b","version":"114'357","reported_seq":4306,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.271533+0000","last_change":"2024-09-19T21:36:56.272264+0000","last_active":"2024-09-19T21:36:56.271533+0000","last_peered":"2024-09-19T21:36:56.271533+0000","last_clean":"2024-09-19T21:36:56.271533+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:36:56.271533+0000","last_undegraded":"2024-09-19T21:36:56.271533+0000","last_fullsized":"2024-09-19T21:36:56.271533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909839,"stat_sum":{"num_bytes":180001481,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":24,"num_read":4137,"num_read_kb":25855,"num_write":347,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"2"}]},{"pgid":"3.1a","version":"122'912","reported_seq":8394,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.505438+0000","last_change":"2024-09-19T21:36:57.288425+0000","last_active":"2024-09-19T21:36:57.505438+0000","last_peered":"2024-09-19T21:36:57.505438+0000","last_clean":"2024-09-19T21:36:57.505438+0000","last_became_active":"2024-09-19T21:36:05.848564+0000","last_became_peered":"2024-09-19T21:36:05.848564+0000","last_unstale":"2024-09-19T21:36:57.505438+0000","last_undegraded":"2024-09-19T21:36:57.505438+0000","last_fullsized":"2024-09-19T21:36:57.505438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.523212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915100000000005,"stat_sum":{"num_bytes":49103002,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":19,"num_read":11398,"num_read_kb":37736,"num_write":1402,"num_write_kb":8624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":4294810,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.19","version":"122'356","reported_seq":14364,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.385779+0000","last_change":"2024-09-19T21:36:57.332915+0000","last_active":"2024-09-19T21:36:57.385779+0000","last_peered":"2024-09-19T21:36:57.385779+0000","last_clean":"2024-09-19T21:36:57.385779+0000","last_became_active":"2024-09-19T21:34:30.006156+0000","last_became_peered":"2024-09-19T21:34:30.006156+0000","last_unstale":"2024-09-19T21:36:57.385779+0000","last_undegraded":"2024-09-19T21:36:57.385779+0000","last_fullsized":"2024-09-19T21:36:57.385779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:38:18.084516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077560699999999999,"stat_sum":{"num_bytes":42093762,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":14774,"num_read_kb":34058,"num_write":440,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.18","version":"119'270","reported_seq":10156,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.272196+0000","last_change":"2024-09-19T21:36:56.272196+0000","last_active":"2024-09-19T21:36:56.272196+0000","last_peered":"2024-09-19T21:36:56.272196+0000","last_clean":"2024-09-19T21:36:56.272196+0000","last_became_active":"2024-09-19T21:34:30.007241+0000","last_became_peered":"2024-09-19T21:34:30.007241+0000","last_unstale":"2024-09-19T21:36:56.272196+0000","last_undegraded":"2024-09-19T21:36:56.272196+0000","last_fullsized":"2024-09-19T21:36:56.272196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:13:47.252736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020006609999999999,"stat_sum":{"num_bytes":84483684,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":10648,"num_read_kb":42213,"num_write":343,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"2"}]},{"pgid":"3.17","version":"119'237","reported_seq":6045,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.280881+0000","last_change":"2024-09-19T21:36:57.281371+0000","last_active":"2024-09-19T21:36:57.280881+0000","last_peered":"2024-09-19T21:36:57.280881+0000","last_clean":"2024-09-19T21:36:57.280881+0000","last_became_active":"2024-09-19T21:36:05.848182+0000","last_became_peered":"2024-09-19T21:36:05.848182+0000","last_unstale":"2024-09-19T21:36:57.280881+0000","last_undegraded":"2024-09-19T21:36:57.280881+0000","last_fullsized":"2024-09-19T21:36:57.280881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:18.525133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001200095,"stat_sum":{"num_bytes":53030400,"num_objects":75,"num_object_clones":28,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":5999,"num_read_kb":14628,"num_write":199,"num_write_kb":4534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.16","version":"121'289","reported_seq":10783,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.282947+0000","last_change":"2024-09-19T21:36:57.283686+0000","last_active":"2024-09-19T21:36:57.282947+0000","last_peered":"2024-09-19T21:36:57.282947+0000","last_clean":"2024-09-19T21:36:57.282947+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:36:57.282947+0000","last_undegraded":"2024-09-19T21:36:57.282947+0000","last_fullsized":"2024-09-19T21:36:57.282947+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014986369999999999,"stat_sum":{"num_bytes":74823720,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":11750,"num_read_kb":30030,"num_write":241,"num_write_kb":6838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.15","version":"122'342","reported_seq":6691,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.504692+0000","last_change":"2024-09-19T21:36:57.282508+0000","last_active":"2024-09-19T21:36:57.504692+0000","last_peered":"2024-09-19T21:36:57.504692+0000","last_clean":"2024-09-19T21:36:57.504692+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:36:57.504692+0000","last_undegraded":"2024-09-19T21:36:57.504692+0000","last_fullsized":"2024-09-19T21:36:57.504692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001851159,"stat_sum":{"num_bytes":15141667,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":6601,"num_read_kb":12028,"num_write":274,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.14","version":"121'284","reported_seq":9808,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.288528+0000","last_change":"2024-09-19T21:36:57.288817+0000","last_active":"2024-09-19T21:36:57.288528+0000","last_peered":"2024-09-19T21:36:57.288528+0000","last_clean":"2024-09-19T21:36:57.288528+0000","last_became_active":"2024-09-19T21:36:36.059875+0000","last_became_peered":"2024-09-19T21:36:36.059875+0000","last_unstale":"2024-09-19T21:36:57.288528+0000","last_undegraded":"2024-09-19T21:36:57.288528+0000","last_fullsized":"2024-09-19T21:36:57.288528+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:30:00.883700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056259099999999998,"stat_sum":{"num_bytes":50334224,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":31,"num_read":9639,"num_read_kb":27148,"num_write":247,"num_write_kb":4768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":6472736,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":118,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.277801+0000","last_change":"2024-09-19T21:34:22.953808+0000","last_active":"2024-09-19T21:36:56.277801+0000","last_peered":"2024-09-19T21:36:56.277801+0000","last_clean":"2024-09-19T21:36:56.277801+0000","last_became_active":"2024-09-19T21:34:20.955938+0000","last_became_peered":"2024-09-19T21:34:20.955938+0000","last_unstale":"2024-09-19T21:36:56.277801+0000","last_undegraded":"2024-09-19T21:36:56.277801+0000","last_fullsized":"2024-09-19T21:36:56.277801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:57:21.462158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001091116,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"122'445","reported_seq":8308,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.333965+0000","last_change":"2024-09-19T21:36:57.333965+0000","last_active":"2024-09-19T21:36:57.333965+0000","last_peered":"2024-09-19T21:36:57.333965+0000","last_clean":"2024-09-19T21:36:57.333965+0000","last_became_active":"2024-09-19T21:34:30.005909+0000","last_became_peered":"2024-09-19T21:34:30.005909+0000","last_unstale":"2024-09-19T21:36:57.333965+0000","last_undegraded":"2024-09-19T21:36:57.333965+0000","last_fullsized":"2024-09-19T21:36:57.333965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:30:59.535198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013842349999999999,"stat_sum":{"num_bytes":71374466,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":8572,"num_read_kb":23369,"num_write":400,"num_write_kb":6471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.282040+0000","last_change":"2024-09-19T21:34:22.956831+0000","last_active":"2024-09-19T21:36:56.282040+0000","last_peered":"2024-09-19T21:36:56.282040+0000","last_clean":"2024-09-19T21:36:56.282040+0000","last_became_active":"2024-09-19T21:34:20.963972+0000","last_became_peered":"2024-09-19T21:34:20.963972+0000","last_unstale":"2024-09-19T21:36:56.282040+0000","last_undegraded":"2024-09-19T21:36:56.282040+0000","last_fullsized":"2024-09-19T21:36:56.282040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:06:33.449917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073442799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"121'672","reported_seq":8461,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.332967+0000","last_change":"2024-09-19T21:36:57.333649+0000","last_active":"2024-09-19T21:36:57.332967+0000","last_peered":"2024-09-19T21:36:57.332967+0000","last_clean":"2024-09-19T21:36:57.332967+0000","last_became_active":"2024-09-19T21:34:30.006290+0000","last_became_peered":"2024-09-19T21:34:30.006290+0000","last_unstale":"2024-09-19T21:36:57.332967+0000","last_undegraded":"2024-09-19T21:36:57.332967+0000","last_fullsized":"2024-09-19T21:36:57.332967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:35.097129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131419,"stat_sum":{"num_bytes":45459706,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":18,"num_read":10415,"num_read_kb":41946,"num_write":765,"num_write_kb":10710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.277569+0000","last_change":"2024-09-19T21:34:22.954340+0000","last_active":"2024-09-19T21:36:56.277569+0000","last_peered":"2024-09-19T21:36:56.277569+0000","last_clean":"2024-09-19T21:36:56.277569+0000","last_became_active":"2024-09-19T21:34:20.956594+0000","last_became_peered":"2024-09-19T21:34:20.956594+0000","last_unstale":"2024-09-19T21:36:56.277569+0000","last_undegraded":"2024-09-19T21:36:56.277569+0000","last_fullsized":"2024-09-19T21:36:56.277569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:07:35.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"121'434","reported_seq":12289,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.332487+0000","last_change":"2024-09-19T21:36:57.332667+0000","last_active":"2024-09-19T21:36:57.332487+0000","last_peered":"2024-09-19T21:36:57.332487+0000","last_clean":"2024-09-19T21:36:57.332487+0000","last_became_active":"2024-09-19T21:36:36.058617+0000","last_became_peered":"2024-09-19T21:36:36.058617+0000","last_unstale":"2024-09-19T21:36:57.332487+0000","last_undegraded":"2024-09-19T21:36:57.332487+0000","last_fullsized":"2024-09-19T21:36:57.332487+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:26:34.274961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049187500000000002,"stat_sum":{"num_bytes":75941594,"num_objects":75,"num_object_clones":24,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":32,"num_read":12330,"num_read_kb":41215,"num_write":341,"num_write_kb":7012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":15059605,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-19T21:36:55.262528+0000","last_change":"2024-09-19T21:36:32.030602+0000","last_active":"2024-09-19T21:36:55.262528+0000","last_peered":"2024-09-19T21:36:55.262528+0000","last_clean":"2024-09-19T21:36:55.262528+0000","last_became_active":"2024-09-19T21:36:32.030278+0000","last_became_peered":"2024-09-19T21:36:32.030278+0000","last_unstale":"2024-09-19T21:36:55.262528+0000","last_undegraded":"2024-09-19T21:36:55.262528+0000","last_fullsized":"2024-09-19T21:36:55.262528+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:24:40.163359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"122'791","reported_seq":5250,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.282426+0000","last_change":"2024-09-19T21:36:57.282426+0000","last_active":"2024-09-19T21:36:57.282426+0000","last_peered":"2024-09-19T21:36:57.282426+0000","last_clean":"2024-09-19T21:36:57.282426+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:36:57.282426+0000","last_undegraded":"2024-09-19T21:36:57.282426+0000","last_fullsized":"2024-09-19T21:36:57.282426+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057789699999999996,"stat_sum":{"num_bytes":41241820,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":8135,"num_read_kb":22342,"num_write":1014,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.3","version":"122'732","reported_seq":9127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.462229+0000","last_change":"2024-09-19T21:36:57.280687+0000","last_active":"2024-09-19T21:36:57.462229+0000","last_peered":"2024-09-19T21:36:57.462229+0000","last_clean":"2024-09-19T21:36:57.462229+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:36:57.462229+0000","last_undegraded":"2024-09-19T21:36:57.462229+0000","last_fullsized":"2024-09-19T21:36:57.462229+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001012619,"stat_sum":{"num_bytes":59795765,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":12886,"num_read_kb":20809,"num_write":798,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.2","version":"21'2","reported_seq":144,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.384074+0000","last_change":"2024-09-19T21:36:02.839832+0000","last_active":"2024-09-19T21:36:56.384074+0000","last_peered":"2024-09-19T21:36:56.384074+0000","last_clean":"2024-09-19T21:36:56.384074+0000","last_became_active":"2024-09-19T21:36:02.830895+0000","last_became_peered":"2024-09-19T21:36:02.830895+0000","last_unstale":"2024-09-19T21:36:56.384074+0000","last_undegraded":"2024-09-19T21:36:56.384074+0000","last_fullsized":"2024-09-19T21:36:56.384074+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:24:40.111547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"119'452","reported_seq":11490,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.283266+0000","last_change":"2024-09-19T21:36:57.283480+0000","last_active":"2024-09-19T21:36:57.283266+0000","last_peered":"2024-09-19T21:36:57.283266+0000","last_clean":"2024-09-19T21:36:57.283266+0000","last_became_active":"2024-09-19T21:36:38.061093+0000","last_became_peered":"2024-09-19T21:36:38.061093+0000","last_unstale":"2024-09-19T21:36:57.283266+0000","last_undegraded":"2024-09-19T21:36:57.283266+0000","last_fullsized":"2024-09-19T21:36:57.283266+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:38:17.668473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001284561,"stat_sum":{"num_bytes":36375145,"num_objects":52,"num_object_clones":13,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":12600,"num_read_kb":49967,"num_write":439,"num_write_kb":12192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":17020731,"num_keys_recovered":94,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-19T21:36:55.260783+0000","last_change":"2024-09-19T21:36:02.831334+0000","last_active":"2024-09-19T21:36:55.260783+0000","last_peered":"2024-09-19T21:36:55.260783+0000","last_clean":"2024-09-19T21:36:55.260783+0000","last_became_active":"2024-09-19T21:36:02.830988+0000","last_became_peered":"2024-09-19T21:36:02.830988+0000","last_unstale":"2024-09-19T21:36:55.260783+0000","last_undegraded":"2024-09-19T21:36:55.260783+0000","last_fullsized":"2024-09-19T21:36:55.260783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:09:35.907712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"122'658","reported_seq":8598,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.332351+0000","last_change":"2024-09-19T21:36:57.332351+0000","last_active":"2024-09-19T21:36:57.332351+0000","last_peered":"2024-09-19T21:36:57.332351+0000","last_clean":"2024-09-19T21:36:57.332351+0000","last_became_active":"2024-09-19T21:36:35.056705+0000","last_became_peered":"2024-09-19T21:36:35.056705+0000","last_unstale":"2024-09-19T21:36:57.332351+0000","last_undegraded":"2024-09-19T21:36:57.332351+0000","last_fullsized":"2024-09-19T21:36:57.332351+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:15:07.237414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036122599999999997,"stat_sum":{"num_bytes":84418228,"num_objects":75,"num_object_clones":24,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":28,"num_read":9714,"num_read_kb":35296,"num_write":923,"num_write_kb":10405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":13548106,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.276282+0000","last_change":"2024-09-19T21:36:32.033895+0000","last_active":"2024-09-19T21:36:56.276282+0000","last_peered":"2024-09-19T21:36:56.276282+0000","last_clean":"2024-09-19T21:36:56.276282+0000","last_became_active":"2024-09-19T21:36:32.033629+0000","last_became_peered":"2024-09-19T21:36:32.033629+0000","last_unstale":"2024-09-19T21:36:56.276282+0000","last_undegraded":"2024-09-19T21:36:56.276282+0000","last_fullsized":"2024-09-19T21:36:56.276282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:52:25.528924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"121'387","reported_seq":12388,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.272830+0000","last_change":"2024-09-19T21:36:56.272830+0000","last_active":"2024-09-19T21:36:56.272830+0000","last_peered":"2024-09-19T21:36:56.272830+0000","last_clean":"2024-09-19T21:36:56.272830+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:36:56.272830+0000","last_undegraded":"2024-09-19T21:36:56.272830+0000","last_fullsized":"2024-09-19T21:36:56.272830+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024225340000000001,"stat_sum":{"num_bytes":136467521,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":18,"num_read":13267,"num_read_kb":48374,"num_write":423,"num_write_kb":3493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"2"}]},{"pgid":"2.3","version":"19'1","reported_seq":143,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.277219+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:36:56.277219+0000","last_peered":"2024-09-19T21:36:56.277219+0000","last_clean":"2024-09-19T21:36:56.277219+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:36:56.277219+0000","last_undegraded":"2024-09-19T21:36:56.277219+0000","last_fullsized":"2024-09-19T21:36:56.277219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":329,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.277743+0000","last_change":"2024-09-19T21:36:37.060633+0000","last_active":"2024-09-19T21:36:56.277743+0000","last_peered":"2024-09-19T21:36:56.277743+0000","last_clean":"2024-09-19T21:36:56.277743+0000","last_became_active":"2024-09-19T21:36:37.060036+0000","last_became_peered":"2024-09-19T21:36:37.060036+0000","last_unstale":"2024-09-19T21:36:56.277743+0000","last_undegraded":"2024-09-19T21:36:56.277743+0000","last_fullsized":"2024-09-19T21:36:56.277743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:41:47.787667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"121'443","reported_seq":15281,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.332755+0000","last_change":"2024-09-19T21:36:57.333407+0000","last_active":"2024-09-19T21:36:57.332755+0000","last_peered":"2024-09-19T21:36:57.332755+0000","last_clean":"2024-09-19T21:36:57.332755+0000","last_became_active":"2024-09-19T21:36:37.059116+0000","last_became_peered":"2024-09-19T21:36:37.059116+0000","last_unstale":"2024-09-19T21:36:57.332755+0000","last_undegraded":"2024-09-19T21:36:57.332755+0000","last_fullsized":"2024-09-19T21:36:57.332755+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:14:10.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089879800000000002,"stat_sum":{"num_bytes":113242200,"num_objects":88,"num_object_clones":26,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":32,"num_read":14268,"num_read_kb":62274,"num_write":523,"num_write_kb":13066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":17942192,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.9","version":"114'376","reported_seq":5444,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.279554+0000","last_change":"2024-09-19T21:36:57.279994+0000","last_active":"2024-09-19T21:36:57.279554+0000","last_peered":"2024-09-19T21:36:57.279554+0000","last_clean":"2024-09-19T21:36:57.279554+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:36:57.279554+0000","last_undegraded":"2024-09-19T21:36:57.279554+0000","last_fullsized":"2024-09-19T21:36:57.279554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000871184,"stat_sum":{"num_bytes":103381759,"num_objects":62,"num_object_clones":19,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":21,"num_read":5809,"num_read_kb":18693,"num_write":352,"num_write_kb":6846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.a","version":"122'501","reported_seq":8187,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.280068+0000","last_change":"2024-09-19T21:36:57.280068+0000","last_active":"2024-09-19T21:36:57.280068+0000","last_peered":"2024-09-19T21:36:57.280068+0000","last_clean":"2024-09-19T21:36:57.280068+0000","last_became_active":"2024-09-19T21:34:30.001936+0000","last_became_peered":"2024-09-19T21:34:30.001936+0000","last_unstale":"2024-09-19T21:36:57.280068+0000","last_undegraded":"2024-09-19T21:36:57.280068+0000","last_fullsized":"2024-09-19T21:36:57.280068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:20:09.433901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000796789,"stat_sum":{"num_bytes":36311860,"num_objects":70,"num_object_clones":18,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":27,"num_read":9564,"num_read_kb":47491,"num_write":598,"num_write_kb":13316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.b","version":"119'265","reported_seq":2312,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-19T21:36:56.272394+0000","last_change":"2024-09-19T21:36:56.272903+0000","last_active":"2024-09-19T21:36:56.272394+0000","last_peered":"2024-09-19T21:36:56.272394+0000","last_clean":"2024-09-19T21:36:56.272394+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:36:56.272394+0000","last_undegraded":"2024-09-19T21:36:56.272394+0000","last_fullsized":"2024-09-19T21:36:56.272394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021042259999999998,"stat_sum":{"num_bytes":22356612,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":13,"num_read":2290,"num_read_kb":21223,"num_write":292,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"2"}]},{"pgid":"3.c","version":"119'255","reported_seq":3691,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.280801+0000","last_change":"2024-09-19T21:36:57.281056+0000","last_active":"2024-09-19T21:36:57.280801+0000","last_peered":"2024-09-19T21:36:57.280801+0000","last_clean":"2024-09-19T21:36:57.280801+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:36:57.280801+0000","last_undegraded":"2024-09-19T21:36:57.280801+0000","last_fullsized":"2024-09-19T21:36:57.280801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013000279999999999,"stat_sum":{"num_bytes":17011712,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":3572,"num_read_kb":13410,"num_write":214,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.d","version":"122'411","reported_seq":9385,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.433205+0000","last_change":"2024-09-19T21:36:57.288871+0000","last_active":"2024-09-19T21:36:57.433205+0000","last_peered":"2024-09-19T21:36:57.433205+0000","last_clean":"2024-09-19T21:36:57.433205+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:36:57.433205+0000","last_undegraded":"2024-09-19T21:36:57.433205+0000","last_fullsized":"2024-09-19T21:36:57.433205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055661399999999998,"stat_sum":{"num_bytes":121945344,"num_objects":84,"num_object_clones":27,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":31,"num_read":9401,"num_read_kb":32097,"num_write":452,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.e","version":"119'508","reported_seq":6049,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.282725+0000","last_change":"2024-09-19T21:36:57.283180+0000","last_active":"2024-09-19T21:36:57.282725+0000","last_peered":"2024-09-19T21:36:57.282725+0000","last_clean":"2024-09-19T21:36:57.282725+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:36:57.282725+0000","last_undegraded":"2024-09-19T21:36:57.282725+0000","last_fullsized":"2024-09-19T21:36:57.282725+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011190550000000001,"stat_sum":{"num_bytes":33208372,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":8111,"num_read_kb":51590,"num_write":435,"num_write_kb":7397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.f","version":"121'726","reported_seq":4382,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.281130+0000","last_change":"2024-09-19T21:36:57.281364+0000","last_active":"2024-09-19T21:36:57.281130+0000","last_peered":"2024-09-19T21:36:57.281130+0000","last_clean":"2024-09-19T21:36:57.281130+0000","last_became_active":"2024-09-19T21:34:30.000097+0000","last_became_peered":"2024-09-19T21:34:30.000097+0000","last_unstale":"2024-09-19T21:36:57.281130+0000","last_undegraded":"2024-09-19T21:36:57.281130+0000","last_fullsized":"2024-09-19T21:36:57.281130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:31.025549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088078199999999998,"stat_sum":{"num_bytes":10553711,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":6982,"num_read_kb":14016,"num_write":815,"num_write_kb":4922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.10","version":"121'488","reported_seq":8033,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.287718+0000","last_change":"2024-09-19T21:36:57.287985+0000","last_active":"2024-09-19T21:36:57.287718+0000","last_peered":"2024-09-19T21:36:57.287718+0000","last_clean":"2024-09-19T21:36:57.287718+0000","last_became_active":"2024-09-19T21:36:05.848208+0000","last_became_peered":"2024-09-19T21:36:05.848208+0000","last_unstale":"2024-09-19T21:36:57.287718+0000","last_undegraded":"2024-09-19T21:36:57.287718+0000","last_fullsized":"2024-09-19T21:36:57.287718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:25:57.433432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042517500000000002,"stat_sum":{"num_bytes":47015561,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":35,"num_read":9697,"num_read_kb":59172,"num_write":445,"num_write_kb":18037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":13819481,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.11","version":"120'358","reported_seq":3866,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.282043+0000","last_change":"2024-09-19T21:36:57.282283+0000","last_active":"2024-09-19T21:36:57.282043+0000","last_peered":"2024-09-19T21:36:57.282043+0000","last_clean":"2024-09-19T21:36:57.282043+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:36:57.282043+0000","last_undegraded":"2024-09-19T21:36:57.282043+0000","last_fullsized":"2024-09-19T21:36:57.282043+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060051400000000002,"stat_sum":{"num_bytes":80246308,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":4409,"num_read_kb":20492,"num_write":267,"num_write_kb":2915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.12","version":"122'251","reported_seq":7208,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.280531+0000","last_change":"2024-09-19T21:36:57.280531+0000","last_active":"2024-09-19T21:36:57.280531+0000","last_peered":"2024-09-19T21:36:57.280531+0000","last_clean":"2024-09-19T21:36:57.280531+0000","last_became_active":"2024-09-19T21:36:05.846322+0000","last_became_peered":"2024-09-19T21:36:05.846322+0000","last_unstale":"2024-09-19T21:36:57.280531+0000","last_undegraded":"2024-09-19T21:36:57.280531+0000","last_fullsized":"2024-09-19T21:36:57.280531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:34.465745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011879010000000001,"stat_sum":{"num_bytes":76729344,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":19,"num_read":6850,"num_read_kb":72074,"num_write":269,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":2561536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"3"}]},{"pgid":"3.13","version":"118'329","reported_seq":7515,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-19T21:36:57.287806+0000","last_change":"2024-09-19T21:36:57.288055+0000","last_active":"2024-09-19T21:36:57.287806+0000","last_peered":"2024-09-19T21:36:57.287806+0000","last_clean":"2024-09-19T21:36:57.287806+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:36:57.287806+0000","last_undegraded":"2024-09-19T21:36:57.287806+0000","last_fullsized":"2024-09-19T21:36:57.287806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040399799999999998,"stat_sum":{"num_bytes":34933412,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":7954,"num_read_kb":20587,"num_write":349,"num_write_kb":5723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1987258140,"num_objects":1976,"num_object_clones":545,"num_object_copies":3952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1976,"num_whiteouts":730,"num_read":294000,"num_read_kb":1051769,"num_write":16604,"num_write_kb":268707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2810,"num_bytes_recovered":173104204,"num_keys_recovered":458,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":443351040,"data_stored":587277132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15165,"ondisk_log_size":15165,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":0,"seq":55834574885,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95700,"kb_used_data":68668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94276140,"statfs":{"total":96636764160,"available":96538767360,"internally_reserved":0,"allocated":70316032,"data_stored":90806643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90200000000000002,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88100000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.73599999999999999,"15min":0.745},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.169},"last":0.75},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80900000000000005,"15min":0.82099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0640000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.92300000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73899999999999999},"min":{"1min":0.41499999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.004,"5min":1.048,"15min":1.048},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.46200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71699999999999997,"15min":0.71999999999999997},"min":{"1min":0.35399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.98399999999999999,"5min":0.996,"15min":0.996},"last":0.622},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.81999999999999995,"15min":0.82399999999999995},"min":{"1min":0.44400000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.024,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.86799999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:36:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.79900000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.25,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.86099999999999999},"min":{"1min":0.41399999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2889999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128032,"kb_used_data":66580,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94243808,"statfs":{"total":96636764160,"available":96505659392,"internally_reserved":0,"allocated":68177920,"data_stored":81494689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48999999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.626}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.69699999999999995,"15min":0.69099999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4330000000000001,"5min":1.921,"15min":1.921},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.879,"15min":0.879},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.626,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.89500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133292,"kb_used_data":80028,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94238548,"statfs":{"total":96636764160,"available":96500273152,"internally_reserved":0,"allocated":81948672,"data_stored":112148028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.66700000000000004,"15min":0.63300000000000001},"min":{"1min":0.41499999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78500000000000003,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.81200000000000006},"min":{"1min":0.51600000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":5.0739999999999998,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86499999999999999,"15min":0.85299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":5.0369999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.92300000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.266,"5min":1.282,"15min":1.282},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.86599999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.0990000000000002,"5min":3.0990000000000002,"15min":3.0990000000000002},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":128100,"kb_used_data":83028,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94243740,"statfs":{"total":96636764160,"available":96505589760,"internally_reserved":0,"allocated":85020672,"data_stored":120869627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.96099999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.31,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83699999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.87,"5min":1.333,"15min":1.333},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.012,"15min":1.012},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.47699999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83799999999999997,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0900000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.754}]},{"osd":4,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.874,"15min":0.89000000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0740000000000001,"5min":1.718,"15min":1.718},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.02,"5min":1.248,"15min":1.248},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81100000000000005,"15min":0.81299999999999994},"min":{"1min":0.57299999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.71999999999999997}]}]},{"osd":3,"up_from":12,"seq":51539607588,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":142464,"kb_used_data":89204,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94229376,"statfs":{"total":96636764160,"available":96490881024,"internally_reserved":0,"allocated":91344896,"data_stored":112799490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77100000000000002,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.38400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92000000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40999999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.026,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.79100000000000004,"15min":0.80000000000000004},"min":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0269999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.997}]},{"osd":2,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":3.2669999999999999,"5min":1.175,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":42.331000000000003,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":3.2719999999999998,"5min":1.2330000000000001,"15min":0.89300000000000002},"min":{"1min":0.504,"5min":0.379,"15min":0.379},"max":{"1min":42.182000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.99399999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.50700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.105,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.46800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.86699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":83578880,"data_stored":120280399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":79912960,"data_stored":110968407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":66146304,"data_stored":80315112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":89702400,"data_stored":112005679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":68710400,"data_stored":90077579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":55300096,"data_stored":73629956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:36:59.122 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [3, 1, 0, 5] 2024-09-19T21:36:59.122 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1', '3', '1', '0', '5'] 2024-09-19T21:36:59.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1 3 1 0 5 2024-09-19T21:36:59.217 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:59.210+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:59.277 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.277 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:59.277 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:59.277 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:36:59.317 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:59.310+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:59.399 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.399 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:59.400 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:59.400 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:59.400 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:36:59.418 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:59.414+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:59.519 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:36:59.514+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:36:59.524 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.524 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.322534026, 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:59.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321954114999997, 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:36:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321891468000004, 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:36:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321825148000002, 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:36:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321121469999994, 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321048648000001, 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:36:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:36:59.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:59.552 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321034909999995, 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:59.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:59.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.321017221000005, 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.558 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:36:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:36:59.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.320966834999993, 2024-09-19T21:36:59.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.320939945999996, 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.320899370000006, 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.572 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:36:59.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.320877756000002, 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:36:59.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:36:59.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 91.320858556999994, 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:36:59.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:36:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.831540954999994, 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:36:59.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.830917473, 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 77.830585197000005, 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:36:59.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:36:59.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.741493980000001, 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:36:59.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:36:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:36:59.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.718897625, 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:36:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.718566958, 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:36:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.718047385999999, 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:36:59.616 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.621 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:36:59.623 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:36:59.618+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:36:59.724 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:59.722+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:59.736 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.736 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:59.736 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:36:59.736 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:36:59.824 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:36:59.818+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:36:59.852 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.852 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:36:59.852 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:36:59.852 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:36:59.852 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:36:59.862 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:36:59.925 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:36:59.922+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 124.222024894, 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:36:59.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:36:59.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594685498000004, 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:36:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:36:59.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594633032000004, 2024-09-19T21:36:59.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:36:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:36:59.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594602484999996, 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.991 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:36:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:36:59.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594579887999998, 2024-09-19T21:36:59.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:36:59.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:36:59.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:36:59.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594557436000002, 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:00.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:00.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594543139999999, 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:00.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594526513000005, 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:00.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:00.014 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594507876999998, 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:00.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594481212000005, 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.019 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:00.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.023 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594471001000002, 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:00.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.594458514999999, 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.028 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:00.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:00.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590583925000004, 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:37:00.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:37:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:00.037 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590566405999994, 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.041 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590528852000006, 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:00.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:00.046 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590511008999997, 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:00.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.59049856, 2024-09-19T21:37:00.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:00.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590487867999997, 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:00.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:00.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.060 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.590476291000002, 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:00.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 79.174504334999995, 2024-09-19T21:37:00.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:00.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:00.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:00.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:00.071 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:00.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:00.073 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:00.066+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:00.173 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:00.166+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:00.188 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:00.188 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:00.188 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:00.188 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:00.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:00.274 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:00.270+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:00.307 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:00.307 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:00.307 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:00.307 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:00.307 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:00.320 INFO:teuthology.orchestra.run.smithi012.stderr:set 3.1 pg_upmap_items mapping to [3->1,0->5] 2024-09-19T21:37:00.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:37:00.325 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:00.379 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:00.374+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:00.476 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:00.470+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.877305557, 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:00.499 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:00.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.877283829000007, 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:00.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.877263096999997, 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:00.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.512 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.877228661000004, 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:00.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.877185863999998, 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:00.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871960353999995, 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:00.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:00.527 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871938056000005, 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:00.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871907200999999, 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:00.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:00.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.537 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871864492, 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:00.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:00.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871840504999994, 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871809100999997, 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:00.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871783934999996, 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871738856999997, 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:00.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871712113000001, 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:00.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:00.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871698820999995, 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 87.871681902000006, 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.571 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:00.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 28.471695130000001, 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:00.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:00.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 28.470128426999999, 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:00.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:00.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:00.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:00.587 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 28.470092146999999, 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:00.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:00.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:00.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 28.470072125000002, 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:00.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:00.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:00.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:00.599 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:00.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:00.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:00.602 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:00.598+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:00.701 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:00.694+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:00.709 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:00.709 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:00.709 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:00.709 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:00.719 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:00.736 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:00.756 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:00.774 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:00.798 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:00.802 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:00.798+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:00.820 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:00.820 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:00.822 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:00.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:00.847 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:00.849 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:00.850 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:00.906 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:00.902+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:00.936 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:00.936 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:00.946 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:01.005 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:00.998+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:01.051 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.051 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:01.106 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:01.102+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:01.167 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.167 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.177 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:01.206 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:01.202+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:01.290 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.290 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.300 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:01.307 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:01.302+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:01.339 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:37:01.408 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:01.402+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:01.422 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.422 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:01.508 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:01.502+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:01.538 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.539 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:01.609 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:01.602+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:01.710 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:01.706+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:01.773 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.773 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:01.829 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:01.826+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:01.894 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:01.894 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:01.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:01.912 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:01.906+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:02.009 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.009 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.012 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:02.006+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:02.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:02.114 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:02.110+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:02.168 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.169 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:02.214 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:02.210+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:02.287 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.287 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:02.341 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:02.338+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:02.415 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:02.410+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:02.446 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.446 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:02.517 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:02.510+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:02.583 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.583 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.595 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:02.617 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:02.614+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:02.701 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.701 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:02.717 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:02.714+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:02.818 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:02.814+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:02.819 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.819 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:02.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:02.919 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:02.914+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:02.937 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:02.938 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:02.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:03.013 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:03.019 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:03.014+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:03.037 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:03.059 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:03.061 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:03.061 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:03.061 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:03.062 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:03.062 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:03.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:03.080 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:03.103 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:03.120 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:03.114+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:03.126 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:03.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 167.23940232000001, 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:03.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:03.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919834988999995, 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.183 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:03.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919823222999995, 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.188 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.190 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:03.192 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919810330000004, 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.193 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:03.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:03.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919798306999994, 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:03.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:03.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:03.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:03.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:03.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:03.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:03.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919780662999997, 2024-09-19T21:37:03.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:03.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:03.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:03.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:03.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:03.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919764866999998, 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:03.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:03.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:03.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919753016000001, 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:03.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:03.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.225 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919684172000004, 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:03.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:03.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:03.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919656032000006, 2024-09-19T21:37:03.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:03.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:03.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.236 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.919632902999993, 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:03.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.904856283000001, 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:03.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:03.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.897519967999997, 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:03.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.250 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.897499077999996, 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.251 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:03.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.890423537999993, 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.257 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:03.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.258 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:03.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:03.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.263 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.890402993999999, 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.710048923000002, 2024-09-19T21:37:03.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:03.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:03.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:03.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:03.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:03.274 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.710038495999996, 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.275 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.710028066999996, 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:03.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:03.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:03.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 90.710018004000005, 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.289 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:03.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:03.291 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:03.286+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:03.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:37:03.392 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:03.386+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:03.451 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:03.451 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:03.451 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:03.451 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:03.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:03.493 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:03.490+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:03.558 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:03.558 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:03.559 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:03.559 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:03.559 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:03.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:03.618 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:03.614+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:03.674 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.941463102, 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:03.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:03.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:03.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.339507858000005, 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:03.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:03.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:03.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.339061876000002, 2024-09-19T21:37:03.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.688 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:03.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:03.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338947696999995, 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:03.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:03.698 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338931263999996, 2024-09-19T21:37:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.699 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:03.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:03.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338915708000002, 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338801638999996, 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338784676000003, 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:03.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:03.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:03.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:03.720 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338768692000002, 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:03.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:03.725 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:03.726 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338751517000006, 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:03.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.731 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338734318999997, 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:03.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:03.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.737 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338717488, 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:03.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338700966999994, 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:03.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:03.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:03.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338678363, 2024-09-19T21:37:03.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:03.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:03.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:03.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338661974999994, 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:03.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.757 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:03.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:03.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.760 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.760 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:03.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 77.338553094000005, 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.765 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:03.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.767 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.659522058999997, 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:03.768 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:03.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:03.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:03.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:03.774 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.659408910000003, 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:03.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.650143397000001, 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:03.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:03.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 32.650070972000002, 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:03.786 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:03.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:03.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:03.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:03.791 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:03.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:03.793 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:03.786+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:03.849 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:37:03.893 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:03.890+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:03.926 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:03.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:03.926 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:03.926 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:03.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:03.995 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:03.990+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:04.045 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.045 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:04.045 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:04.045 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:04.045 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:04.095 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:04.090+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90716214699999, 2024-09-19T21:37:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907148608, 2024-09-19T21:37:04.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907122868, 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:04.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.180 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907108654, 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:04.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.185 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907087737, 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:04.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:04.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907072361, 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90705743300001, 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:04.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.907004813, 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.204 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.906955235, 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:04.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:04.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90693802, 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:04.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.906918882, 2024-09-19T21:37:04.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:04.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90690403399999, 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:04.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:04.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90688104100001, 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.224 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:04.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.226 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:04.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.906148749, 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.229 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:04.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.233 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.906098439, 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.235 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90603612699999, 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:04.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:04.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:04.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.904705494, 2024-09-19T21:37:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:04.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:04.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.904569916, 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:04.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.904493157, 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.90443443300001, 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:04.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.264 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.265 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:04.266 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:04.262+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:04.367 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:04.362+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.423 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:04.423 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:04.423 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:04.467 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:04.466+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:04.540 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.540 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:04.540 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:04.540 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:04.541 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:04.568 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:04.562+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.461094924999998, 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.460515013999995, 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:04.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.460452367000002, 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:04.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.675 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.460386047, 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:04.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459682369000006, 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:04.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459609546999999, 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459595809000007, 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:04.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459578120000003, 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.699 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459527734000005, 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:04.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:04.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:04.704 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459500844999994, 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459460269000004, 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.713 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.714 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459438655, 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:04.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.459419456000006, 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:04.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:04.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:04.723 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 82.970101854000006, 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:04.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:04.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 82.969478371999998, 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.732 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 82.969146096000003, 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.737 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.880054878999999, 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.739 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:04.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:04.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:04.743 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.857458523999998, 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.745 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:04.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.857127857000002, 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.856608285, 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:04.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.756 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:04.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:04.760 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:04.762 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:04.758+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:04.862 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:04.858+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:04.922 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:04.922 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:04.922 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:04.923 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:04.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:04.962 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:04.958+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:05.047 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:05.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:05.047 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:05.048 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:05.048 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:05.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:05.064 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:05.058+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:05.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:05.165 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 129.41214475300001, 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:05.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:05.171 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784805356999996, 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:05.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784752890999997, 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:05.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784722344000002, 2024-09-19T21:37:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:05.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784699747000005, 2024-09-19T21:37:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:05.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784677294999995, 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:05.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784662999000005, 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784646371999997, 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784627736000004, 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.208 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784601070999997, 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:05.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784590859999994, 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.216 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.217 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.784578374000006, 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:05.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:05.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:05.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17110 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3715072~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780703783999996, 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065149951999999997, 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17110 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380063+0000", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380062+0000", 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380065+0000", 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.667e-06 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380071+0000", 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9429999999999997e-06 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380076+0000", 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8550000000000001e-06 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444919+0000", 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.064842957000000007 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.444988+0000", 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9610999999999995e-05 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445213+0000", 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00022431999999999999 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780686265, 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:05.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:05.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780648710999998, 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:05.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.234 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780630868000003, 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.235 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:05.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:05.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:05.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780618419000007, 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:05.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:05.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780607727000003, 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:05.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:05.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.780596149999994, 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:05.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:05.250 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:05.162+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:05.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 84.364624194000001, 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.254 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:05.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:05.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:05.258 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:05.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:05.265 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:05.258+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:05.309 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:05.336 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:05.336 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:05.336 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-19T21:37:05.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-19T21:37:05.343 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:05.361 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:05.366 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:05.362+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:05.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:05.385 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:05.385 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:05.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:05.411 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:05.436 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:05.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:05.466 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:05.462+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:05.517 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:05.518 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:05.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:05.567 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:05.562+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:05.639 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:05.639 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:05.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:05.667 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:05.662+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:05.670 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:37:05.662+0000 7eff847dd700 -1 mon.a@0(leader).osd e130 definitely_dead 0 2024-09-19T21:37:05.755 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:05.756 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:05.768 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:05.762+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:05.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:05.860 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.5. 2024-09-19T21:37:05.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:37:05.873 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-19T21:37:05.873 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:05.866+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:05.890 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:05.890 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:05.900 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:05.970 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:05.966+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:06.052 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.052 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.062 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:06.070 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:06.066+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:06.167 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.167 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.171 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:06.166+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:06.177 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:06.272 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:06.266+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:06.283 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.284 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:06.372 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:06.366+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:06.401 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.401 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:06.473 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:06.466+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:06.532 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.532 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:06.574 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:06.570+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:06.651 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.651 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:06.675 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:06.670+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:06.771 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.771 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.775 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:06.770+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:06.780 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:06.863 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:37:06.891 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:06.892 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:06.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:06.918 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:06.914+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:06.977 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:06.974+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:07.012 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:07.013 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:07.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:07.078 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:07.070+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:07.178 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:07.178 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:07.179 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:07.174+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:07.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:07.280 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:07.274+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:07.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:07.381 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:07.378+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:07.412 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:07.412 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:07.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:07.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:07.482 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:07.478+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:07.533 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:07.533 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:07.543 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:07.589 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:07.586+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:07.648 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:07.661 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:07.661 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:07.662 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:07.662 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:07.666 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:07.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:07.683 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:07.678+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:07.689 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:07.712 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:07.736 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:07.738 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:07.777 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:07.777 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:07.777 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:07.777 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:07.777 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:07.784 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:07.778+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:07.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:07.884 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:07.878+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:07.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.316316053999998, 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:07.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.944 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.316294326000005, 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:07.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:07.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.316273593999995, 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:07.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:07.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.316239158000002, 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:07.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.958 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.316196360999996, 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:07.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:07.963 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310970850999993, 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:07.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:07.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310948553000003, 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:07.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:07.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:07.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310917697999997, 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:07.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:07.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.977 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310874988999998, 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:07.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:07.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310851002000007, 2024-09-19T21:37:07.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:07.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:07.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:07.988 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310819597999995, 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:07.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:07.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310794431999994, 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.993 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:07.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310749353999995, 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:07.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310722609999999, 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:08.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:08.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.006 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310709317999994, 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:08.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.310692399000004, 2024-09-19T21:37:08.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:08.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:08.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.910705626999999, 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:08.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.021 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.909138923999997, 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:08.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:08.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.909102644000001, 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:08.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:08.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.036 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 35.909082622, 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:08.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:08.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.042 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:08.044 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:08.038+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:08.144 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:08.142+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:08.203 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.203 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:08.203 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:08.203 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.213 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:08.245 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:08.238+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:08.339 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.339 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:08.339 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:08.339 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:08.339 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.346 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:08.342+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:08.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:08.447 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:08.442+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:08.509 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:08.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 172.57273178599999, 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.510 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:08.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:08.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:08.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.517 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253164455000004, 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:08.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:08.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253152689000004, 2024-09-19T21:37:08.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:08.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:08.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.527 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253139795999999, 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:08.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:08.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.532 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253127773000003, 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253110129000007, 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:08.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253094333000007, 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.543 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:08.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:08.547 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253082481999996, 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:08.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:08.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.253013637999999, 2024-09-19T21:37:08.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:08.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:08.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.252985498000001, 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.557 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:08.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:08.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.252962369000002, 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:08.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:08.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.238185748999996, 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:08.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:08.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.230849434000007, 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.230828544000005, 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.223753004000002, 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.223732459999994, 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.043378388999997, 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.043367962000005, 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.043357533000005, 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 96.04334747, 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.613 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:08.615 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:08.610+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:08.715 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:08.710+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:08.736 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:08.736 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:08.737 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.747 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:08.816 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:08.810+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:08.860 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:08.860 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:08.861 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:08.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:08.876 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:37:08.917 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:08.914+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:08.989 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:08.989 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 133.25872101100001, 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:08.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:08.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:08.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:08.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:08.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656765766999996, 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:08.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:08.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:09.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:09.002 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656319784999994, 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.003 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:09.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656205606, 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656189173000001, 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656173616999993, 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.018 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:09.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.022 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656059548000002, 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:09.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656042584999994, 2024-09-19T21:37:09.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:09.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656026600999994, 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.032 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:09.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:09.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.656009425999997, 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:09.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:09.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655992228000002, 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:09.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655975397000006, 2024-09-19T21:37:09.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:09.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:09.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655958876, 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.051 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:09.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.055 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655936272000005, 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:09.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:09.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655919883999999, 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:09.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 82.655811002999997, 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:09.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:09.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 37.976779968000002, 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:09.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:09.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:09.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:09.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 37.976666819000002, 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:09.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:09.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 37.967401305999999, 2024-09-19T21:37:09.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:09.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:09.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:09.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:09.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:09.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:09.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.095 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 37.967328881, 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:09.096 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:09.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:09.102 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:09.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:09.104 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:09.098+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:09.204 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:09.198+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:09.233 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.233 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:09.233 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:09.234 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:09.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:09.305 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:09.302+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:09.353 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.353 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:09.353 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:09.353 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:09.353 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:09.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:09.406 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:09.402+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:09.508 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:09.502+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.527 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268190805, 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:09.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.532 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268177266, 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.533 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:09.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.537 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268151526, 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:09.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.542 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26813731199999, 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:09.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26811639500001, 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:09.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268101019, 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:09.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268086091, 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.556 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:09.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.560 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.268033471, 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.561 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:09.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26798389299999, 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:09.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26796667799999, 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:09.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26794753999999, 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.267932692, 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:09.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.267909699, 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:09.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:09.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26717740700001, 2024-09-19T21:37:09.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:09.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:09.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.267127097, 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.267064785, 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:09.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26573415199999, 2024-09-19T21:37:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.26559857399999, 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:09.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.265521815, 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:09.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:09.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:09.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.265463091, 2024-09-19T21:37:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:09.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.628 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:09.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:09.630 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:09.622+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:09.731 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:09.726+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:09.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:09.754 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.754 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:09.754 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:09.755 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:09.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:09.831 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:09.826+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:09.870 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:09.871 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:09.871 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:09.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:09.891 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:37:09.913 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:09.932 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:09.926+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:09.936 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:09.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:09.978 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:09.988 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.786551297, 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.785971386, 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:09.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:09.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.78590873900001, 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:10.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:10.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.78584241900001, 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.785138741, 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:10.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.785065919, 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.018 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:10.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.020 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:10.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.785052181, 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:10.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.78503449199999, 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:10.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.784984106, 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.784957217, 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.038 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:10.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:10.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.042 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.042 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.042 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.784916641, 2024-09-19T21:37:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:10.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.048 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.784895027, 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:10.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.784875828, 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:10.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 88.295558225999997, 2024-09-19T21:37:10.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:10.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:10.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 88.294934744000003, 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:10.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:10.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 88.294602467999994, 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.205511250999997, 2024-09-19T21:37:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:10.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.182914896, 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.078 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:10.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.182584229, 2024-09-19T21:37:10.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.088 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 33.182064656999998, 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:10.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:10.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:10.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:10.094 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:10.094 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:10.095 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:10.095 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:10.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:10.097 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:10.090+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:10.097 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:10.097 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:10.098 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:10.098 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:10.098 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:10.098 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:10.196 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:10.190+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:10.267 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.268 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:10.297 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:10.294+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:10.384 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.384 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:10.437 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:10.434+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:10.498 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:10.494+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:10.552 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.552 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:10.598 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:10.594+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:10.679 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.679 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:10.700 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:10.694+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:10.795 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.795 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.801 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:10.794+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:10.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:10.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 2, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:10.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:10.873 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 3, 4, 2, 5] 2024-09-19T21:37:10.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-19T21:37:10.901 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:10.898+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:10.921 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:10.921 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:10.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:11.002 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:10.998+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:11.057 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.058 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:11.103 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:11.098+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:11.169 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:11.200 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:37:11.194+0000 7eff847dd700 -1 mon.a@0(leader).osd e136 definitely_dead 0 2024-09-19T21:37:11.203 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:11.198+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:11.304 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:11.298+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:11.307 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.307 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:11.405 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:11.398+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:11.424 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.424 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:11.506 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:11.498+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:11.588 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.588 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.598 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:11.606 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:11.602+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:11.705 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.705 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.707 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:11.702+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:11.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:11.808 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:11.806+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:11.832 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.832 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.4. 2024-09-19T21:37:11.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:37:11.909 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:11.902+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:11.960 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:11.960 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:11.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:12.010 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:12.006+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:12.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:12.183 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:12.184 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:12.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:12.184 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:12.178+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:12.284 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:12.278+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:12.302 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:12.302 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:12.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:12.323 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:12.351 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:12.371 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:12.385 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:12.378+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:12.395 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:12.419 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:12.443 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:12.445 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:12.446 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:12.472 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:12.473 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:12.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:12.486 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:12.478+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:12.586 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:12.582+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.38978530999999, 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.38920539900001, 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.389142752, 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.389076432, 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388372754, 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388299932, 2024-09-19T21:37:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388286194, 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388268505, 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388218119, 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.38819123, 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388150654, 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.38812904, 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 104.388109841, 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 90.898792239000002, 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 90.898168756999993, 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 90.897836480999999, 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.808745264000002, 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.786148908999998, 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.785818241999998, 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 35.785298670000003, 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:12.689 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:12.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:12.691 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:12.686+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:12.791 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:12.786+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:12.844 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:37:12.860 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:12.860 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:12.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:12.902 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:12.898+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:12.973 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:12.973 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:12.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:12.992 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:12.986+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 137.336919309, 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709579912999999, 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709527446999999, 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709496900000005, 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709474302999993, 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709451850999997, 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709437554999994, 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709420928, 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709402291999993, 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709375627, 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709365415999997, 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.709352929999994, 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705460821000003, 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705423267, 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705405424000006, 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705392974999995, 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705382283000006, 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.705370705999997, 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 92.289398750000004, 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 3.9425545959999999, 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:13.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:13.189 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:13.182+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:13.289 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:13.286+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:13.307 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:13.307 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:13.307 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:13.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:13.390 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:13.386+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:13.423 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:13.423 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:13.423 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:13.423 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:13.423 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:13.433 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:13.490 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:13.486+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.92286095599999, 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.546 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:13.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:13.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.922839228, 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.551 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:13.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.555 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.922818496, 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:13.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.92278406, 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.560 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:13.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.92274126300001, 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:13.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:13.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917515753, 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:13.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.574 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917493455, 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:13.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.91746259999999, 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.579 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:13.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.91741989099999, 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.584 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.588 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917395904, 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:13.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.91736450000001, 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:13.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:13.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917339334, 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.597 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.91729425600001, 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:13.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:13.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:13.605 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917267512, 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:13.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.91725422, 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 100.917237301, 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.614 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:13.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.517250529000002, 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:13.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:13.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.515683826, 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:13.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:13.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:13.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.515647545999997, 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:13.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 41.515627524000003, 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:13.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:13.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:13.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:13.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:13.642 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:13.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:13.644 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:13.638+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:13.745 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:13.738+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:13.754 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:13.754 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:13.754 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:13.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:13.846 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:13.842+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:13.895 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:13.895 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:13.895 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:13.895 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:13.895 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:13.906 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:13.946 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:13.942+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:14.032 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:14.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 178.09782678299999, 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.033 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:14.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:14.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:14.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:14.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:14.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778259452, 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:14.046 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778247686, 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:14.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.052 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778234793, 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:14.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.77822277, 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:14.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:14.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778205126, 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:14.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:14.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:14.069 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.77818933, 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:14.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.074 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.77817747899999, 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:14.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778108635, 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:14.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:14.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778080495, 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:14.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:14.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.778057366, 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:14.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.76328074600001, 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:14.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:14.102 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.755944431, 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:14.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.755923541, 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.748848001, 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:14.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:14.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:14.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:14.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.748827457, 2024-09-19T21:37:14.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:14.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:14.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:14.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.124 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.56847338599999, 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.125 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:14.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:14.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:14.130 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.568462959, 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:14.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.56845253, 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:14.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.568442467, 2024-09-19T21:37:14.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:14.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.147 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:14.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:14.149 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:14.142+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:14.249 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:14.242+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:14.321 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:14.321 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDPGPushReply(3.1 139/138 [PushReplyOp(3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14)])", 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:37:14.320166+0000", 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0010538850000000001, 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010891609999999999, 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320166+0000", 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320166+0000", 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320165+0000", 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.323 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320177+0000", 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2233e-05 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320208+0000", 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1352000000000001e-05 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320222+0000", 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4246999999999999e-05 2024-09-19T21:37:14.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:14.320289+0000", 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6848999999999996e-05 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:37:14.325 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:14.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:14.349 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:14.342+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:14.440 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:14.440 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:14.440 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:14.450 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:14.451 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:14.442+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:14.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:14.550 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:14.546+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:14.616 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:14.616 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 138.88388466500001, 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.617 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:14.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281929421000001, 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:14.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.627 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281483438999999, 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:14.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281369260000005, 2024-09-19T21:37:14.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:14.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281352827000006, 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:14.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281337270999998, 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281223202000007, 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:14.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:14.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.650 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281206238999999, 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:14.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:14.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:14.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:14.655 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281190254999999, 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:14.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281173080000002, 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.660 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:14.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.664 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281155881999993, 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:14.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281139050999997, 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:14.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:14.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281122530000005, 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.674 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.678 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281099925999996, 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.281083538000004, 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:14.683 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:14.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.687 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 88.280974657000002, 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:14.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 43.601943622, 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.693 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:14.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:14.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.697 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 43.601830473, 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:14.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:14.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:14.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 43.592564959999997, 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.703 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:14.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:14.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:14.708 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 43.592492534999998, 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:14.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:14.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:14.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:14.714 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:14.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:14.716 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:14.716 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:14.716 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:14.716 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:14.717 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:14.710+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:14.727 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:14.751 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:14.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:14.815 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:14.810+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:14.838 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:14.838 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:14.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:14.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:37:14.915 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:14.910+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:14.968 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:14.969 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:14.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:15.025 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:15.022+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:15.118 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:15.114+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.137 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:15.218 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:15.214+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:15.253 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.253 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:15.320 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:15.314+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:15.369 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.369 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.378 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:15.421 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:15.418+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:15.483 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.483 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:15.522 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:15.518+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:15.598 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.598 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:15.649 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:15.646+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:15.724 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:15.718+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:15.760 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.760 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:15.825 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:15.822+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:15.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:15.915 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:37:15.925 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:15.918+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:16.002 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.003 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:16.026 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:16.022+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:16.126 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.126 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.127 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:16.122+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:16.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:16.228 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.222+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.243 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.243 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:16.328 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.322+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.366 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.366 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:16.430 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.426+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.480 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.480 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:16.530 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.526+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.600 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.600 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:16.631 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.626+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.716 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.716 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:16.732 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:16.726+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:16.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:16.831 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.831 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.833 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:16.826+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:16.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:16.912 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:16.912 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:16.912 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-19T21:37:16.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-19T21:37:16.914 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:16.934 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:16.930+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:16.940 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:16.961 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:16.961 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:16.964 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:16.971 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:16.989 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:17.012 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:17.034 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:17.030+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:17.040 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:17.042 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:17.094 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:17.094 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:17.094 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:17.135 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:17.130+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:17.210 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:17.211 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:17.211 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:17.265 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:17.262+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:17.328 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.328 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:17.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:17.328 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.07023229000001, 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070218751, 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.334 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.336 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:17.337 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.338 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070193011, 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:17.339 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.342 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070178797, 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.345 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.346 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.347 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.07015788, 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.349 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:17.352 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070142504, 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:17.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070127576, 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.360 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070074956, 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.364 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:17.365 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.366 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.368 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070025378, 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.369 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:17.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.070008163, 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:17.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.069989025, 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:17.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.06997417700001, 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:17.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.387 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.069951184, 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.392 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.06921889199999, 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.069168582, 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.06910627000001, 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:17.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.067775637, 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:17.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.411 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.067640059, 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:17.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:17.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.0675633, 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:17.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:17.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.421 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 123.067504576, 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:17.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:17.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.427 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.427 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:17.429 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:17.422+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:17.529 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:17.522+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:17.541 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:17.630 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:17.626+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:17.659 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:17.659 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:17.731 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:17.730+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.575499378, 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.779 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:17.780 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:17.782 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.574919467, 2024-09-19T21:37:17.783 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.784 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.57485681999999, 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.788 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.789 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:17.790 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.791 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.792 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.57479050000001, 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.793 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.794 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:17.795 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.574086822, 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.799 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.801 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.802 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.57401400000001, 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.803 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.805 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.574000262, 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:17.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:17.810 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.812 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.57398257299999, 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.813 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.815 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:17.816 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.817 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.573932187, 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.820 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:17.821 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:17.822 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.573905298, 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:17.823 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.824 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:17.825 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:17.826 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.573864722, 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:17.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.57384310800001, 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.833 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.835 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.573823909, 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:17.838 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.840 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:17.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.084506306999998, 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:17.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:17.845 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.847 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.083882825000003, 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:17.848 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:17.850 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.851 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.852 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 96.083550548999995, 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:17.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.856 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.857 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.994459331999998, 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:17.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.860 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.861 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.971862977000001, 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.865 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.867 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.868 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.971532310000001, 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:17.872 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 40.971012737999999, 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:17.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.876 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.880 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.881 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:17.881 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:17.881 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:moved 4 / 82 (4.87805%) 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:avg 16.4 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.72764 -> 2.87054 (expected baseline 3.62215) 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.5 with 12 -> 13 pgs (0.731707 -> 0.792683 * mean) 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 20 -> 21 pgs (1.21951 -> 1.28049 * mean) 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 3 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:37:17.883 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0011 2024-09-19T21:37:17.884 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0012 2024-09-19T21:37:17.884 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 weight 1.0000 -> 0.8208 2024-09-19T21:37:17.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:17.884 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-19T21:37:17.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-19T21:37:17.885 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:17.878+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:17.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:37:17.983 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:17.978+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:18.002 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:18.002 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:18.002 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:18.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:18.083 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:18.078+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:18.117 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:18.117 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:18.117 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:18.118 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:18.118 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:18.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:18.184 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:18.178+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:18.235 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 142.481666698, 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:18.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:18.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854327302000002, 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:18.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854274836000002, 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:18.249 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.250 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854244288999993, 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.251 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:18.252 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.253 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854221691999996, 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.256 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.85419924, 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854184943999996, 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.265 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:18.266 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.267 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854168317000003, 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.272 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:18.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854149680999996, 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.275 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:18.276 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.277 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.278 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854123016000003, 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:18.280 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.281 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:18.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854112805, 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:18.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.287 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.854100318999997, 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:18.290 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.291 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:18.292 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850208210000005, 2024-09-19T21:37:18.293 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:18.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850170656000003, 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:18.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.299 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:18.300 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.301 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:18.302 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850152812999994, 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:18.305 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.306 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:18.307 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850140363999998, 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:18.308 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.309 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.310 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:18.311 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850129671999994, 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.312 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:18.313 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.315 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.316 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.850118094999999, 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.317 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:18.319 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.320 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 97.434146139000006, 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.322 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:18.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.324 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:18.325 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 9.0873019849999999, 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:18.326 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:18.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:18.332 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:avg 16.4 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.87054 -> 2.87054 (expected baseline 3.62215) 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.5 with 13 -> 13 pgs (0.792683 -> 0.792683 * mean) 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 21 -> 21 pgs (1.28049 -> 1.28049 * mean) 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:oload 1000 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 1 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:37:18.333 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0011 2024-09-19T21:37:18.334 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0105 2024-09-19T21:37:18.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:18.335 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:37:18.336 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-19T21:37:18.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-19T21:37:18.336 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:18.326+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:18.434 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:18.430+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:23426 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:37:18.001312+0000", 2024-09-19T21:37:18.489 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.48786575100000001, 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48789587000000001, 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for peered", 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 23426 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001312+0000", 2024-09-19T21:37:18.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001312+0000", 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001310+0000", 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001320+0000", 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8949999999999993e-06 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001341+0000", 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0248e-05 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001350+0000", 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3130000000000004e-06 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001428+0000", 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7887999999999997e-05 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:18.001443+0000", 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4921e-05 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.493 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:18.494 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:37:18.494 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:18.499 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:18.535 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:18.530+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:18.605 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:18.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:18.605 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:18.606 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:18.606 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:18.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:18.635 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:18.630+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:moved 0 / 82 (0%) 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:avg 16.4 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.87054 -> 2.87054 (expected baseline 3.62215) 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.5 with 13 -> 13 pgs (0.792683 -> 0.792683 * mean) 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 21 -> 21 pgs (1.28049 -> 1.28049 * mean) 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:oload 1000 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 1 2024-09-19T21:37:18.679 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:37:18.680 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0011 2024-09-19T21:37:18.680 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0105 2024-09-19T21:37:18.683 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:37:18.694 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-19T21:37:18.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.724 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.10135139400001, 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.101329666, 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.730 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:18.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:18.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.101308934, 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:18.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:18.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.101274498, 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:18.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:18.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.101231701, 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:18.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.096006191, 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095983893, 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.755 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.756 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.757 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:18.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.759 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095953038, 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.760 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.761 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.762 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:18.763 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.09591032900001, 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.765 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.766 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.767 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:18.768 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.769 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095886342, 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.770 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.771 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.772 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.773 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.774 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095854938, 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:18.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.776 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:18.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.778 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095829772, 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.779 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.780 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:18.781 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.782 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.783 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095784694, 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.784 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:18.785 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.786 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:18.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.09575795000001, 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:18.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.791 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:18.792 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095744658, 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.793 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:18.794 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.795 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:18.796 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.797 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.095727739, 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.798 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.799 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:18.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.801 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.695740966999999, 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.802 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.803 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.804 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:18.805 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.806 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.807 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.694174263999997, 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.808 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.809 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:18.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.811 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:18.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.813 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.814 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.694137984000001, 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:18.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.817 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:18.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.819 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:18.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.821 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.822 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.694117962, 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:18.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:18.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.825 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.826 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.827 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.828 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.829 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:18.829 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:18.829 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:18.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:18.831 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:18.826+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:18.850 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:37:18.931 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:18.926+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:18.948 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:18.948 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:18.949 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:18.949 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:18.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:19.032 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:19.026+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:19.044 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:37:19.038+0000 7f660afcc700 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-19T21:37:19.044 INFO:teuthology.orchestra.run.smithi012.stdout:moved 2 / 82 (2.43902%) 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:avg 16.4 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.87054 -> 3.72021 (expected baseline 3.62215) 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.5 with 13 -> 13 pgs (0.792683 -> 0.792683 * mean) 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 21 -> 22 pgs (1.28049 -> 1.34146 * mean) 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 3 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0011 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0012 2024-09-19T21:37:19.045 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 weight 0.8208 -> 0.6346 2024-09-19T21:37:19.049 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:37:19.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:19.060 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-19T21:37:19.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-19T21:37:19.074 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:19.075 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:19.075 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:19.075 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:19.075 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:19.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:19.132 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:19.126+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:19.187 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:19.197 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.198 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 183.26226993399999, 2024-09-19T21:37:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:19.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:19.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:19.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:19.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942702603, 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.208 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:19.212 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942690837, 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.213 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:19.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:19.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:19.217 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942677944, 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.218 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:19.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:19.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.222 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942665921, 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:19.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:19.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:19.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942648277, 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:19.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:19.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.234 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942632481, 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:19.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.94262062999999, 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:19.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:19.244 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942551786, 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.245 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:19.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:19.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.249 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942523646, 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:19.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.942500517, 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:19.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:19.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:19.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.92772389700001, 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:19.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:19.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.920387582, 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.264 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:19.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.920366692, 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.269 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:19.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.913291152, 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:19.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.278 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.913270608, 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:19.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:19.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.73291653699999, 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:19.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:19.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.73290611, 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.288 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:19.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:19.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:19.292 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.732895681, 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:19.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.296 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.732885618, 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:19.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:19.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:19.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:19.301 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:19.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:19.304 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:19.304 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:19.304 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:19.304 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:19.305 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:19.306 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:19.298+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:19.367 INFO:tasks.ceph.mgr.x.smithi033.stderr:2024-09-19T21:37:19.362+0000 7f660afcc700 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-19T21:37:19.367 INFO:teuthology.orchestra.run.smithi012.stdout:moved 2 / 82 (2.43902%) 2024-09-19T21:37:19.367 INFO:teuthology.orchestra.run.smithi012.stdout:avg 16.4 2024-09-19T21:37:19.367 INFO:teuthology.orchestra.run.smithi012.stdout:stddev 2.87054 -> 3.72021 (expected baseline 3.62215) 2024-09-19T21:37:19.367 INFO:teuthology.orchestra.run.smithi012.stdout:min osd.5 with 13 -> 13 pgs (0.792683 -> 0.792683 * mean) 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:max osd.0 with 21 -> 22 pgs (1.28049 -> 1.34146 * mean) 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:oload 110 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:max_change 3 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:max_change_osds 4 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:average_utilization 0.0011 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:overload_utilization 0.0012 2024-09-19T21:37:19.368 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 weight 0.8208 -> 0.6346 2024-09-19T21:37:19.371 INFO:teuthology.orchestra.run.smithi012.stderr:no change 2024-09-19T21:37:19.403 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:19.398+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:19.412 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:19.412 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:19.422 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:19.504 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:19.498+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:19.579 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:19.579 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:19.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:19.605 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:19.602+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:19.701 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:19.702 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:19.706 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:19.702+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:19.711 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:19.807 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:19.806+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:19.829 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:19.829 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:19.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:19.907 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:19.902+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:19.945 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:19.945 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:19.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:20.008 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:20.002+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:20.060 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.060 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:20.109 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:20.102+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:20.210 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:20.206+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:20.226 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.226 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:20.311 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:20.306+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:20.348 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.348 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:20.412 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:20.406+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:20.480 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.480 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:20.513 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:20.506+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:20.594 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.594 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:20.649 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:20.646+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:20.707 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.707 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.714 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:20.710+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:20.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:20.815 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:20.810+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:20.821 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.821 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.831 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:20.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:37:20.916 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:20.910+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:20.958 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:20.959 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:20.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:21.017 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:21.014+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:21.073 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:21.073 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:21.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:21.117 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:21.114+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:21.187 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:21.187 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:21.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:21.219 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:21.214+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:21.300 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:21.300 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:21.309 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:21.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:21.319 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:21.314+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:21.419 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:21.419 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:21.420 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:21.414+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:21.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:21.486 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:21.507 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:21.522 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:21.518+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:21.526 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:21.534 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:21.534 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:21.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 185.600134973, 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:21.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:21.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:21.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:21.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28056764199999, 2024-09-19T21:37:21.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:21.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:21.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28055587599999, 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:21.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:21.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.551 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.280542983, 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:21.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:21.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.555 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.556 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28053095999999, 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:21.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:21.561 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.280513316, 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:21.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28049752, 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:21.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.280485669, 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:21.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:21.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:21.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.280416825, 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:21.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28038868500001, 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:21.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:21.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.585 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.28036555600001, 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.265588936, 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:21.594 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.258252621, 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.595 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:21.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:21.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.599 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.258231731, 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:21.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.604 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.25115619100001, 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:21.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:21.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.251135647, 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.614 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.070781576, 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.619 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.070771149, 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:21.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.07076072, 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.624 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:21.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:21.628 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 109.070750657, 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:21.633 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:21.633 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:21.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:21.635 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:21.635 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:21.635 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:21.636 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:21.637 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:21.637 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:21.630+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:21.736 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:21.730+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:21.794 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:21.794 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:21.794 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:21.795 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:21.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:21.837 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:21.830+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:21.916 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:21.916 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:21.917 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:21.917 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:21.917 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:21.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:21.937 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:21.934+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:37:22.038 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:22.034+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:22.046 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:22.046 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:22.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:22.046 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 146.314742395, 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:22.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:22.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:22.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712787151000001, 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712341168999998, 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:22.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.062 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712226990000005, 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712210557000006, 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.068 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:22.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:22.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712195000999998, 2024-09-19T21:37:22.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:22.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:22.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712080932000006, 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:22.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:22.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712063968999999, 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:22.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:22.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:22.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712047984999998, 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:22.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.091 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712030810000002, 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.092 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.096 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.712013612000007, 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:22.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.711996780999996, 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:22.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:22.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:22.105 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.711980260000004, 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:22.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:22.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.711957655999996, 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:22.110 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.711941268000004, 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.115 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 95.711832387000001, 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:22.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:22.123 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.032801352, 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:22.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:22.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:22.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:22.128 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.032688202999999, 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:22.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:22.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.023422689999997, 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.133 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:22.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:22.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 51.023350264999998, 2024-09-19T21:37:22.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.140 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:22.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:22.143 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:22.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:22.145 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:22.142+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:22.246 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:22.242+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:22.265 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:22.265 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:22.265 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:22.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:22.346 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:22.342+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:22.385 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:22.385 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:22.385 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:22.386 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:22.386 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:22.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:22.448 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:22.442+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:22.505 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:22.505 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.247078966, 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.247065427, 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24703968700001, 2024-09-19T21:37:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24702547300001, 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.520 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.521 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:22.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.523 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24700455600001, 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.526 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.528 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.529 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24698918000001, 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.530 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.533 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.246974252, 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:22.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.537 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.538 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24692163200001, 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.540 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.543 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24687205399999, 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.544 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.546 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24685483900001, 2024-09-19T21:37:22.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:22.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24683570100001, 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:22.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:22.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.557 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.246820853, 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:22.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24679785999999, 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:22.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:22.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:22.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24606556800001, 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:22.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:22.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.246015258, 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.572 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:22.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.576 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24595294599999, 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:22.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.581 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24462231300001, 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:22.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24448673500001, 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:22.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:22.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.24440997599999, 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.591 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 128.244351252, 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:22.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:22.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:22.600 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:22.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:22.602 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:22.598+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:22.702 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:22.698+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:22.759 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:22.759 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:22.759 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:22.759 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:22.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:22.803 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:22.802+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:22.875 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:22.876 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:22.876 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:22.876 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:22.876 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:22.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:22.903 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:22.898+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:23.005 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:23.005 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.80329589199999, 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:23.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:23.010 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.80271598100001, 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:23.013 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.014 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.802653334, 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:23.018 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.019 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.802587014, 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.020 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:23.022 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.023 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.024 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801883336, 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:23.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:23.028 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801810514, 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.029 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.032 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.033 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801796776, 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.035 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801779087, 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.038 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:23.040 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.042 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801728701, 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:23.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.048 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801701812, 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.049 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.050 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:23.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:23.052 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801661236, 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.053 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:23.054 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.056 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:23.057 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801639622, 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.058 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:23.060 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:23.061 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.801620423, 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:23.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.312302821, 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.067 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:23.069 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.31167933899999, 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:23.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 101.311347063, 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:23.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.222255846000003, 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:23.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.199659490999998, 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:23.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:23.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.199328823999998, 2024-09-19T21:37:23.091 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.093 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:23.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 46.198809251999997, 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:23.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.102 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:23.103 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:22.998+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:23.104 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:23.145 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:23.142+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:23.207 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:23.202+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:23.262 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:23.262 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:23.262 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:23.262 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:23.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:23.307 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:23.302+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:23.379 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:23.379 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:23.379 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:23.379 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:23.379 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:23.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:23.408 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:23.406+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:23.509 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:23.506+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 147.79347156599999, 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.548 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:23.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:23.553 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16613217, 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.166079704, 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:23.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.560 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:23.562 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.166049157, 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.563 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.565 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16602656000001, 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:23.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:23.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.166004108, 2024-09-19T21:37:23.572 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:23.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16598981200001, 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.581 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.165973185, 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:23.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16595454900001, 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.586 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.165927884, 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:23.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:23.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.165917673, 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16590518700001, 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:23.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:23.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.162013078, 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.161975524, 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:23.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:23.614 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.161957681, 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16194523199999, 2024-09-19T21:37:23.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16193454, 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:23.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:23.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 103.16192296299999, 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.632 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 102.745951007, 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:23.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 14.399106852999999, 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:23.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:23.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:23.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:23.644 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:23.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:23.646 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:23.642+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:23.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:23.746 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:23.742+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:23.760 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:23.760 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:23.760 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:23.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:23.789 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:23.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:23.828 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:23.847 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:23.842+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:23.850 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:23.872 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:23.872 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:23.873 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:23.882 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:23.897 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:23.899 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:23.900 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:23.947 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:23.942+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:23.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:37:24.014 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.014 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:24.048 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:24.042+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:24.149 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:24.142+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:24.152 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.153 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:24.250 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:24.246+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:24.313 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.313 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.323 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:24.351 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:24.346+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:24.382 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:24.382 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:24.383 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T21:37:24.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T21:37:24.426 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.427 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:24.478 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:24.474+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:24.554 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:24.550+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:24.579 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.579 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.589 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:24.654 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:24.650+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 149, 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "5c98f4ba-835c-4398-91b1-e251a78b0f2b", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-09-19T21:34:03.067364+0000", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-09-19T21:37:23.946195+0000", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-09-19T21:34:12.817626+0000", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-09-19T21:37:11.878003+0000", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5806080, 2024-09-19T21:37:24.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "nodeep-scrub", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "noscrub", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-09-19T21:37:24.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 6, 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "jewel", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "reef", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:14.862212+0000", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:37:24.695 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-09-19T21:37:24.696 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "17", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:37:24.697 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:37:24.698 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:37:24.699 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.700 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 6.059999942779541, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 6.059999942779541, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.33000001311302185, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:19.884271+0000", 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:37:24.702 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:37:24.703 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "21", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-09-19T21:37:24.704 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 21, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:37:24.705 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:37:24.706 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 2.2699999809265137, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 2.2699999809265137, 2024-09-19T21:37:24.707 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "pool_client.0", 2024-09-19T21:37:24.708 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-09-19T21:34:28.955849+0000", 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-09-19T21:37:24.709 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 32, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 32, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 32, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 32, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 32, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.710 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "149", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 64, 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 149, 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-09-19T21:37:24.711 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T21:37:24.712 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T21:37:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T21:37:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-09-19T21:37:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-09-19T21:37:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-09-19T21:37:24.713 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-09-19T21:37:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-09-19T21:37:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-09-19T21:37:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-09-19T21:37:24.714 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-09-19T21:37:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-09-19T21:37:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-09-19T21:37:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T21:37:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-09-19T21:37:24.715 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-09-19T21:37:24.716 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 1.5099999904632568, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 1.5099999904632568, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.82999998331069946, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 1.25, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 1.25, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.97057855129241943, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-09-19T21:37:24.717 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb", 2024-09-19T21:37:24.718 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 146, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.720 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.721 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 226723117 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6810/226723117", 2024-09-19T21:37:24.724 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6814/226723117", 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/226723117", 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6818/226723117", 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3", 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.725 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0.8208465576171875, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 137, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.726 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.727 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.729 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1690053836 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.730 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/1690053836", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/1690053836", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/1690053836", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/1690053836", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "585d0e6c-0092-4cdb-ae1a-6260630475d0", 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.731 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 146, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.732 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.734 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3902303328 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.736 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6811/3902303328", 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6815/3902303328", 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6822/3902303328", 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6819/3902303328", 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:37:24.737 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "41e181a0-e2f9-491b-9b3d-80f5e108577c", 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 12, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 141, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6800", 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6801", 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.739 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6802", 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6803", 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.740 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6809", 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6811", 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6804", 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6807", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2014863703 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6801/2014863703", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6803/2014863703", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6811/2014863703", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6807/2014863703", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.743 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0c81c5a4-40c7-43cc-8e53-553d71bebe2b", 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 0, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 135, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.744 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6806", 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6810", 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.745 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6813", 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.746 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6815", 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.747 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6820", 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6821", 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6816", 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6818", 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2893455664 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.749 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6810/2893455664", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6815/2893455664", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6821/2893455664", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6818/2893455664", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "c161c258-bee9-41b5-8257-1ba140f79b9f", 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-09-19T21:37:24.750 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.8234710693359375, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 13, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 140, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6805", 2024-09-19T21:37:24.751 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6808", 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6812", 2024-09-19T21:37:24.752 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6814", 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.753 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6822", 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6823", 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.754 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6817", 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.33:6819", 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 4275333014 2024-09-19T21:37:24.755 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.33:6808/4275333014", 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.33:6814/4275333014", 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.33:6823/4275333014", 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.33:6819/4275333014", 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.756 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.115755+0000", 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.300024+0000", 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-09-19T21:37:24.758 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.859187+0000", 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.759 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:09.118947+0000", 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 65536, 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.197978+0000", 2024-09-19T21:37:24.760 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540138322906710015, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-09-19T21:34:10.229555+0000", 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.761 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-09-19T21:37:24.762 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [ 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "snaps": [ 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 25, 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 6 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.763 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [ 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "snaps": [ 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 25, 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 1 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-09-19T21:37:24.764 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:24.765 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:24.766 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.766 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:24.768 INFO:tasks.thrashosds.thrasher:j is {'epoch': 149, 'fsid': '5c98f4ba-835c-4398-91b1-e251a78b0f2b', 'created': '2024-09-19T21:34:03.067364+0000', 'modified': '2024-09-19T21:37:23.946195+0000', 'last_up_change': '2024-09-19T21:34:12.817626+0000', 'last_in_change': '2024-09-19T21:37:11.878003+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T21:34:14.862212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-19T21:34:19.884271+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-19T21:34:28.955849+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 64, 'snap_epoch': 149, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 0.9705785512924194, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': '66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6810', 'nonce': 226723117}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6814', 'nonce': 226723117}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6821', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 226723117}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 226723117}, {'type': 'v1', 'addr': '172.21.15.12:6818', 'nonce': 226723117}]}, 'public_addr': '172.21.15.12:6810/226723117', 'cluster_addr': '172.21.15.12:6814/226723117', 'heartbeat_back_addr': '172.21.15.12:6823/226723117', 'heartbeat_front_addr': '172.21.15.12:6818/226723117', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3', 'up': 1, 'in': 1, 'weight': 0.8208465576171875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 137, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 1690053836}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 1690053836}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 1690053836}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 1690053836}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 1690053836}]}, 'public_addr': '172.21.15.12:6801/1690053836', 'cluster_addr': '172.21.15.12:6803/1690053836', 'heartbeat_back_addr': '172.21.15.12:6807/1690053836', 'heartbeat_front_addr': '172.21.15.12:6805/1690053836', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '585d0e6c-0092-4cdb-ae1a-6260630475d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 146, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6809', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3902303328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6813', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3902303328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6822', 'nonce': 3902303328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6817', 'nonce': 3902303328}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 3902303328}]}, 'public_addr': '172.21.15.12:6811/3902303328', 'cluster_addr': '172.21.15.12:6815/3902303328', 'heartbeat_back_addr': '172.21.15.12:6822/3902303328', 'heartbeat_front_addr': '172.21.15.12:6819/3902303328', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '41e181a0-e2f9-491b-9b3d-80f5e108577c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 141, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6800', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6801', 'nonce': 2014863703}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6802', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6803', 'nonce': 2014863703}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6809', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6811', 'nonce': 2014863703}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6804', 'nonce': 2014863703}, {'type': 'v1', 'addr': '172.21.15.33:6807', 'nonce': 2014863703}]}, 'public_addr': '172.21.15.33:6801/2014863703', 'cluster_addr': '172.21.15.33:6803/2014863703', 'heartbeat_back_addr': '172.21.15.33:6811/2014863703', 'heartbeat_front_addr': '172.21.15.33:6807/2014863703', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '0c81c5a4-40c7-43cc-8e53-553d71bebe2b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6806', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6810', 'nonce': 2893455664}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6813', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6815', 'nonce': 2893455664}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6820', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6821', 'nonce': 2893455664}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6816', 'nonce': 2893455664}, {'type': 'v1', 'addr': '172.21.15.33:6818', 'nonce': 2893455664}]}, 'public_addr': '172.21.15.33:6810/2893455664', 'cluster_addr': '172.21.15.33:6815/2893455664', 'heartbeat_back_addr': '172.21.15.33:6821/2893455664', 'heartbeat_front_addr': '172.21.15.33:6818/2893455664', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'c161c258-bee9-41b5-8257-1ba140f79b9f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.8234710693359375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 140, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6805', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6808', 'nonce': 4275333014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6812', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6814', 'nonce': 4275333014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6822', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6823', 'nonce': 4275333014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6817', 'nonce': 4275333014}, {'type': 'v1', 'addr': '172.21.15.33:6819', 'nonce': 4275333014}]}, 'public_addr': '172.21.15.33:6808/4275333014', 'cluster_addr': '172.21.15.33:6814/4275333014', 'heartbeat_back_addr': '172.21.15.33:6823/4275333014', 'heartbeat_front_addr': '172.21.15.33:6819/4275333014', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.115755+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.300024+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.859187+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:09.118947+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.197978+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T21:34:10.229555+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T21:37:24.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:37:24.770 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:24.766+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:24.870 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:24.866+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:24.877 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:24.878 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:24.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:24.971 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:24.966+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:24.978 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:37:25.011 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.012 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:25.072 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:25.066+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:25.088 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:25.092 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:37:25.106 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-19T21:37:24.964830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1205433946,"num_objects":1572,"num_object_clones":439,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":654,"num_read":298104,"num_read_kb":1055281,"num_write":17537,"num_write_kb":271325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4541,"num_bytes_recovered":280248355,"num_keys_recovered":645,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16337,"ondisk_log_size":16337,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":539404,"kb_used_data":299312,"kb_used_omap":7,"kb_used_meta":239992,"kb_avail":471319796,"statfs":{"total":483183820800,"available":482631471104,"internally_reserved":0,"allocated":306495488,"data_stored":387529387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":245752032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14362939,"num_objects":-46,"num_object_clones":-11,"num_object_copies":-92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-46,"num_whiteouts":-4,"num_read":613,"num_read_kb":520,"num_write":113,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":15538498,"num_keys_recovered":61,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039031"},"pg_stats":[{"pgid":"3.1f","version":"148'453","reported_seq":4543,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.115384+0000","last_change":"2024-09-19T21:37:22.970519+0000","last_active":"2024-09-19T21:37:23.115384+0000","last_peered":"2024-09-19T21:37:23.115384+0000","last_clean":"2024-09-19T21:37:23.115384+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:37:23.115384+0000","last_undegraded":"2024-09-19T21:37:23.115384+0000","last_fullsized":"2024-09-19T21:37:23.115384+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0049799199999999997,"stat_sum":{"num_bytes":41995310,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":6349,"num_read_kb":38448,"num_write":417,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1e","version":"143'246","reported_seq":2514,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.931772+0000","last_change":"2024-09-19T21:37:20.932020+0000","last_active":"2024-09-19T21:37:20.931772+0000","last_peered":"2024-09-19T21:37:20.931772+0000","last_clean":"2024-09-19T21:37:20.931772+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:37:20.931772+0000","last_undegraded":"2024-09-19T21:37:20.931772+0000","last_fullsized":"2024-09-19T21:37:20.931772+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012814910000000001,"stat_sum":{"num_bytes":13386804,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":1779,"num_read_kb":7797,"num_write":189,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"3.1d","version":"145'1480","reported_seq":10289,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.954777+0000","last_change":"2024-09-19T21:37:22.955142+0000","last_active":"2024-09-19T21:37:22.954777+0000","last_peered":"2024-09-19T21:37:22.954777+0000","last_clean":"2024-09-19T21:37:22.954777+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:37:22.954777+0000","last_undegraded":"2024-09-19T21:37:22.954777+0000","last_fullsized":"2024-09-19T21:37:22.954777+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012214400000000001,"stat_sum":{"num_bytes":59494605,"num_objects":42,"num_object_clones":13,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":17185,"num_read_kb":39279,"num_write":1910,"num_write_kb":15196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1c","version":"148'574","reported_seq":16323,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.121285+0000","last_change":"2024-09-19T21:37:22.978803+0000","last_active":"2024-09-19T21:37:23.121285+0000","last_peered":"2024-09-19T21:37:23.121285+0000","last_clean":"2024-09-19T21:37:23.121285+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:37:23.121285+0000","last_undegraded":"2024-09-19T21:37:23.121285+0000","last_fullsized":"2024-09-19T21:37:23.121285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0066837909999999997,"stat_sum":{"num_bytes":14929972,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":17426,"num_read_kb":26560,"num_write":578,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.1b","version":"140'377","reported_seq":4423,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.930522+0000","last_change":"2024-09-19T21:37:20.930961+0000","last_active":"2024-09-19T21:37:20.930522+0000","last_peered":"2024-09-19T21:37:20.930522+0000","last_clean":"2024-09-19T21:37:20.930522+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:37:20.930522+0000","last_undegraded":"2024-09-19T21:37:20.930522+0000","last_fullsized":"2024-09-19T21:37:20.930522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001703532,"stat_sum":{"num_bytes":121850003,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":4144,"num_read_kb":25858,"num_write":353,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"3.1a","version":"148'953","reported_seq":8940,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.219925+0000","last_change":"2024-09-19T21:37:22.976931+0000","last_active":"2024-09-19T21:37:23.219925+0000","last_peered":"2024-09-19T21:37:23.219925+0000","last_clean":"2024-09-19T21:37:23.219925+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:37:23.219925+0000","last_undegraded":"2024-09-19T21:37:23.219925+0000","last_fullsized":"2024-09-19T21:37:23.219925+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048647730000000002,"stat_sum":{"num_bytes":17689754,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":11741,"num_read_kb":38045,"num_write":1427,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.19","version":"146'364","reported_seq":14653,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.930323+0000","last_change":"2024-09-19T21:37:20.930323+0000","last_active":"2024-09-19T21:37:20.930323+0000","last_peered":"2024-09-19T21:37:20.930323+0000","last_clean":"2024-09-19T21:37:20.930323+0000","last_became_active":"2024-09-19T21:37:15.907615+0000","last_became_peered":"2024-09-19T21:37:15.907615+0000","last_unstale":"2024-09-19T21:37:20.930323+0000","last_undegraded":"2024-09-19T21:37:20.930323+0000","last_fullsized":"2024-09-19T21:37:20.930323+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:01:19.735252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189269,"stat_sum":{"num_bytes":28695218,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":14777,"num_read_kb":34059,"num_write":444,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"3.18","version":"143'288","reported_seq":10465,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.929678+0000","last_change":"2024-09-19T21:37:20.930186+0000","last_active":"2024-09-19T21:37:20.929678+0000","last_peered":"2024-09-19T21:37:20.929678+0000","last_clean":"2024-09-19T21:37:20.929678+0000","last_became_active":"2024-09-19T21:37:15.911405+0000","last_became_peered":"2024-09-19T21:37:15.911405+0000","last_unstale":"2024-09-19T21:37:20.929678+0000","last_undegraded":"2024-09-19T21:37:20.929678+0000","last_fullsized":"2024-09-19T21:37:20.929678+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:59:19.031902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012012909999999999,"stat_sum":{"num_bytes":32263230,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":10652,"num_read_kb":42215,"num_write":349,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"3.17","version":"148'275","reported_seq":6264,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.966747+0000","last_change":"2024-09-19T21:37:22.966747+0000","last_active":"2024-09-19T21:37:22.966747+0000","last_peered":"2024-09-19T21:37:22.966747+0000","last_clean":"2024-09-19T21:37:22.966747+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:37:22.966747+0000","last_undegraded":"2024-09-19T21:37:22.966747+0000","last_fullsized":"2024-09-19T21:37:22.966747+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456555,"stat_sum":{"num_bytes":9508352,"num_objects":59,"num_object_clones":22,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":28,"num_read":6201,"num_read_kb":14803,"num_write":220,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.16","version":"148'343","reported_seq":11116,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.139343+0000","last_change":"2024-09-19T21:37:22.957956+0000","last_active":"2024-09-19T21:37:23.139343+0000","last_peered":"2024-09-19T21:37:23.139343+0000","last_clean":"2024-09-19T21:37:23.139343+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:37:23.139343+0000","last_undegraded":"2024-09-19T21:37:23.139343+0000","last_fullsized":"2024-09-19T21:37:23.139343+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040023339999999998,"stat_sum":{"num_bytes":38614568,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":12237,"num_read_kb":30453,"num_write":278,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.15","version":"146'367","reported_seq":6861,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.952943+0000","last_change":"2024-09-19T21:37:22.953548+0000","last_active":"2024-09-19T21:37:22.952943+0000","last_peered":"2024-09-19T21:37:22.952943+0000","last_clean":"2024-09-19T21:37:22.952943+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:37:22.952943+0000","last_undegraded":"2024-09-19T21:37:22.952943+0000","last_fullsized":"2024-09-19T21:37:22.952943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002282091,"stat_sum":{"num_bytes":12257995,"num_objects":52,"num_object_clones":17,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":6706,"num_read_kb":12121,"num_write":291,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.14","version":"148'322","reported_seq":10227,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.116806+0000","last_change":"2024-09-19T21:37:22.957329+0000","last_active":"2024-09-19T21:37:23.116806+0000","last_peered":"2024-09-19T21:37:23.116806+0000","last_clean":"2024-09-19T21:37:23.116806+0000","last_became_active":"2024-09-19T21:37:21.937346+0000","last_became_peered":"2024-09-19T21:37:21.937346+0000","last_unstale":"2024-09-19T21:37:23.116806+0000","last_undegraded":"2024-09-19T21:37:23.116806+0000","last_fullsized":"2024-09-19T21:37:23.116806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:52:22.072608+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0037486099999999999,"stat_sum":{"num_bytes":35005968,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":27,"num_read":9807,"num_read_kb":27294,"num_write":270,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.934992+0000","last_change":"2024-09-19T21:37:18.848384+0000","last_active":"2024-09-19T21:37:21.934992+0000","last_peered":"2024-09-19T21:37:21.934992+0000","last_clean":"2024-09-19T21:37:21.934992+0000","last_became_active":"2024-09-19T21:37:18.847932+0000","last_became_peered":"2024-09-19T21:37:18.847932+0000","last_unstale":"2024-09-19T21:37:21.934992+0000","last_undegraded":"2024-09-19T21:37:21.934992+0000","last_fullsized":"2024-09-19T21:37:21.934992+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:29.428084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"146'498","reported_seq":8726,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.931984+0000","last_change":"2024-09-19T21:37:20.931984+0000","last_active":"2024-09-19T21:37:20.931984+0000","last_peered":"2024-09-19T21:37:20.931984+0000","last_clean":"2024-09-19T21:37:20.931984+0000","last_became_active":"2024-09-19T21:37:15.907862+0000","last_became_peered":"2024-09-19T21:37:15.907862+0000","last_unstale":"2024-09-19T21:37:20.931984+0000","last_undegraded":"2024-09-19T21:37:20.931984+0000","last_fullsized":"2024-09-19T21:37:20.931984+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.181084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016219310000000001,"stat_sum":{"num_bytes":51721293,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":8743,"num_read_kb":23516,"num_write":429,"num_write_kb":6477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.934791+0000","last_change":"2024-09-19T21:37:12.849531+0000","last_active":"2024-09-19T21:37:21.934791+0000","last_peered":"2024-09-19T21:37:21.934791+0000","last_clean":"2024-09-19T21:37:21.934791+0000","last_became_active":"2024-09-19T21:37:12.849150+0000","last_became_peered":"2024-09-19T21:37:12.849150+0000","last_unstale":"2024-09-19T21:37:21.934791+0000","last_undegraded":"2024-09-19T21:37:21.934791+0000","last_fullsized":"2024-09-19T21:37:21.934791+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:39:56.298451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"142'700","reported_seq":8773,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.931206+0000","last_change":"2024-09-19T21:37:20.931634+0000","last_active":"2024-09-19T21:37:20.931206+0000","last_peered":"2024-09-19T21:37:20.931206+0000","last_clean":"2024-09-19T21:37:20.931206+0000","last_became_active":"2024-09-19T21:37:16.916361+0000","last_became_peered":"2024-09-19T21:37:16.916361+0000","last_unstale":"2024-09-19T21:37:20.931206+0000","last_undegraded":"2024-09-19T21:37:20.931206+0000","last_fullsized":"2024-09-19T21:37:20.931206+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:35:55.526552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017198599999999999,"stat_sum":{"num_bytes":19101761,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":10441,"num_read_kb":41963,"num_write":786,"num_write_kb":10710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.934084+0000","last_change":"2024-09-19T21:37:12.843070+0000","last_active":"2024-09-19T21:37:21.934084+0000","last_peered":"2024-09-19T21:37:21.934084+0000","last_clean":"2024-09-19T21:37:21.934084+0000","last_became_active":"2024-09-19T21:37:12.842703+0000","last_became_peered":"2024-09-19T21:37:12.842703+0000","last_unstale":"2024-09-19T21:37:21.934084+0000","last_undegraded":"2024-09-19T21:37:21.934084+0000","last_fullsized":"2024-09-19T21:37:21.934084+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:10:55.086102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"141'482","reported_seq":12964,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.965591+0000","last_change":"2024-09-19T21:37:22.965591+0000","last_active":"2024-09-19T21:37:22.965591+0000","last_peered":"2024-09-19T21:37:22.965591+0000","last_clean":"2024-09-19T21:37:22.965591+0000","last_became_active":"2024-09-19T21:37:15.909850+0000","last_became_peered":"2024-09-19T21:37:15.909850+0000","last_unstale":"2024-09-19T21:37:22.965591+0000","last_undegraded":"2024-09-19T21:37:22.965591+0000","last_fullsized":"2024-09-19T21:37:22.965591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:22.734267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057216799999999996,"stat_sum":{"num_bytes":38204453,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":27,"num_read":12538,"num_read_kb":41392,"num_write":375,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":510,"num_bytes_recovered":24749279,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T21:37:19.846709+0000","last_change":"2024-09-19T21:37:06.857489+0000","last_active":"2024-09-19T21:37:19.846709+0000","last_peered":"2024-09-19T21:37:19.846709+0000","last_clean":"2024-09-19T21:37:19.846709+0000","last_became_active":"2024-09-19T21:37:06.857170+0000","last_became_peered":"2024-09-19T21:37:06.857170+0000","last_unstale":"2024-09-19T21:37:19.846709+0000","last_undegraded":"2024-09-19T21:37:19.846709+0000","last_fullsized":"2024-09-19T21:37:19.846709+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:21:16.051466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"145'809","reported_seq":5371,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.954259+0000","last_change":"2024-09-19T21:37:22.954665+0000","last_active":"2024-09-19T21:37:22.954259+0000","last_peered":"2024-09-19T21:37:22.954259+0000","last_clean":"2024-09-19T21:37:22.954259+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:37:22.954259+0000","last_undegraded":"2024-09-19T21:37:22.954259+0000","last_fullsized":"2024-09-19T21:37:22.954259+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097960800000000008,"stat_sum":{"num_bytes":40408648,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":20,"num_read":8175,"num_read_kb":22370,"num_write":1031,"num_write_kb":8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.3","version":"148'812","reported_seq":9365,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.150599+0000","last_change":"2024-09-19T21:37:22.950416+0000","last_active":"2024-09-19T21:37:23.150599+0000","last_peered":"2024-09-19T21:37:23.150599+0000","last_clean":"2024-09-19T21:37:23.150599+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:37:23.150599+0000","last_undegraded":"2024-09-19T21:37:23.150599+0000","last_fullsized":"2024-09-19T21:37:23.150599+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086454700000000002,"stat_sum":{"num_bytes":25461029,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":12982,"num_read_kb":20905,"num_write":870,"num_write_kb":4907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.2","version":"21'2","reported_seq":182,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.934701+0000","last_change":"2024-09-19T21:37:12.849222+0000","last_active":"2024-09-19T21:37:21.934701+0000","last_peered":"2024-09-19T21:37:21.934701+0000","last_clean":"2024-09-19T21:37:21.934701+0000","last_became_active":"2024-09-19T21:37:12.846220+0000","last_became_peered":"2024-09-19T21:37:12.846220+0000","last_unstale":"2024-09-19T21:37:21.934701+0000","last_undegraded":"2024-09-19T21:37:21.934701+0000","last_fullsized":"2024-09-19T21:37:21.934701+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:33:18.345371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"147'514","reported_seq":11808,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.965664+0000","last_change":"2024-09-19T21:37:22.966027+0000","last_active":"2024-09-19T21:37:22.965664+0000","last_peered":"2024-09-19T21:37:22.965664+0000","last_clean":"2024-09-19T21:37:22.965664+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:37:22.965664+0000","last_undegraded":"2024-09-19T21:37:22.965664+0000","last_fullsized":"2024-09-19T21:37:22.965664+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082817100000000005,"stat_sum":{"num_bytes":27430468,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":13041,"num_read_kb":50347,"num_write":490,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":158,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T21:37:19.846451+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:37:19.846451+0000","last_peered":"2024-09-19T21:37:19.846451+0000","last_clean":"2024-09-19T21:37:19.846451+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:37:19.846451+0000","last_undegraded":"2024-09-19T21:37:19.846451+0000","last_fullsized":"2024-09-19T21:37:19.846451+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"146'690","reported_seq":8951,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.951679+0000","last_change":"2024-09-19T21:37:22.953054+0000","last_active":"2024-09-19T21:37:22.951679+0000","last_peered":"2024-09-19T21:37:22.951679+0000","last_clean":"2024-09-19T21:37:22.951679+0000","last_became_active":"2024-09-19T21:37:15.910410+0000","last_became_peered":"2024-09-19T21:37:15.910410+0000","last_unstale":"2024-09-19T21:37:22.951679+0000","last_undegraded":"2024-09-19T21:37:22.951679+0000","last_fullsized":"2024-09-19T21:37:22.951679+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:01:14.472862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022406710000000001,"stat_sum":{"num_bytes":57145381,"num_objects":55,"num_object_clones":18,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":9745,"num_read_kb":35320,"num_write":945,"num_write_kb":10410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.932177+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:37:21.932177+0000","last_peered":"2024-09-19T21:37:21.932177+0000","last_clean":"2024-09-19T21:37:21.932177+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:37:21.932177+0000","last_undegraded":"2024-09-19T21:37:21.932177+0000","last_fullsized":"2024-09-19T21:37:21.932177+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"141'427","reported_seq":12604,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.930338+0000","last_change":"2024-09-19T21:37:20.931075+0000","last_active":"2024-09-19T21:37:20.930338+0000","last_peered":"2024-09-19T21:37:20.930338+0000","last_clean":"2024-09-19T21:37:20.930338+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:37:20.930338+0000","last_undegraded":"2024-09-19T21:37:20.930338+0000","last_fullsized":"2024-09-19T21:37:20.930338+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018593010000000001,"stat_sum":{"num_bytes":74998259,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":13473,"num_read_kb":48551,"num_write":450,"num_write_kb":3499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"2.3","version":"19'1","reported_seq":169,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.934081+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:37:21.934081+0000","last_peered":"2024-09-19T21:37:21.934081+0000","last_clean":"2024-09-19T21:37:21.934081+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:37:21.934081+0000","last_undegraded":"2024-09-19T21:37:21.934081+0000","last_fullsized":"2024-09-19T21:37:21.934081+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":376,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-19T21:37:21.933444+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:37:21.933444+0000","last_peered":"2024-09-19T21:37:21.933444+0000","last_clean":"2024-09-19T21:37:21.933444+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:37:21.933444+0000","last_undegraded":"2024-09-19T21:37:21.933444+0000","last_fullsized":"2024-09-19T21:37:21.933444+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"146'461","reported_seq":15670,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.953582+0000","last_change":"2024-09-19T21:37:22.954033+0000","last_active":"2024-09-19T21:37:22.953582+0000","last_peered":"2024-09-19T21:37:22.953582+0000","last_clean":"2024-09-19T21:37:22.953582+0000","last_became_active":"2024-09-19T21:37:15.906980+0000","last_became_peered":"2024-09-19T21:37:15.906980+0000","last_unstale":"2024-09-19T21:37:22.953582+0000","last_undegraded":"2024-09-19T21:37:22.953582+0000","last_fullsized":"2024-09-19T21:37:22.953582+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:23:14.867134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077258100000000005,"stat_sum":{"num_bytes":72898080,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":29,"num_read":14275,"num_read_kb":62277,"num_write":531,"num_write_kb":13066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.9","version":"148'395","reported_seq":5562,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.157576+0000","last_change":"2024-09-19T21:37:22.954717+0000","last_active":"2024-09-19T21:37:23.157576+0000","last_peered":"2024-09-19T21:37:23.157576+0000","last_clean":"2024-09-19T21:37:23.157576+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:37:23.157576+0000","last_undegraded":"2024-09-19T21:37:23.157576+0000","last_fullsized":"2024-09-19T21:37:23.157576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054509110000000001,"stat_sum":{"num_bytes":54738655,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":5818,"num_read_kb":18696,"num_write":359,"num_write_kb":6846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.a","version":"148'552","reported_seq":8556,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.131916+0000","last_change":"2024-09-19T21:37:22.952141+0000","last_active":"2024-09-19T21:37:23.131916+0000","last_peered":"2024-09-19T21:37:23.131916+0000","last_clean":"2024-09-19T21:37:23.131916+0000","last_became_active":"2024-09-19T21:37:21.938850+0000","last_became_peered":"2024-09-19T21:37:21.938850+0000","last_unstale":"2024-09-19T21:37:23.131916+0000","last_undegraded":"2024-09-19T21:37:23.131916+0000","last_fullsized":"2024-09-19T21:37:23.131916+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:20.509636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024409459999999998,"stat_sum":{"num_bytes":28043866,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":9654,"num_read_kb":47562,"num_write":646,"num_write_kb":13326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.b","version":"130'281","reported_seq":2425,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-19T21:37:20.931109+0000","last_change":"2024-09-19T21:37:20.931534+0000","last_active":"2024-09-19T21:37:20.931109+0000","last_peered":"2024-09-19T21:37:20.931109+0000","last_clean":"2024-09-19T21:37:20.931109+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:37:20.931109+0000","last_undegraded":"2024-09-19T21:37:20.931109+0000","last_fullsized":"2024-09-19T21:37:20.931109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020607619999999998,"stat_sum":{"num_bytes":11100760,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":2321,"num_read_kb":21247,"num_write":308,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"4"}]},{"pgid":"3.c","version":"148'271","reported_seq":3822,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.224646+0000","last_change":"2024-09-19T21:37:22.954162+0000","last_active":"2024-09-19T21:37:23.224646+0000","last_peered":"2024-09-19T21:37:23.224646+0000","last_clean":"2024-09-19T21:37:23.224646+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:37:23.224646+0000","last_undegraded":"2024-09-19T21:37:23.224646+0000","last_fullsized":"2024-09-19T21:37:23.224646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002605043,"stat_sum":{"num_bytes":4770816,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":3640,"num_read_kb":13467,"num_write":227,"num_write_kb":4461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.d","version":"148'443","reported_seq":9518,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.975713+0000","last_change":"2024-09-19T21:37:22.975713+0000","last_active":"2024-09-19T21:37:22.975713+0000","last_peered":"2024-09-19T21:37:22.975713+0000","last_clean":"2024-09-19T21:37:22.975713+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:37:22.975713+0000","last_undegraded":"2024-09-19T21:37:22.975713+0000","last_fullsized":"2024-09-19T21:37:22.975713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0037391,"stat_sum":{"num_bytes":51463212,"num_objects":63,"num_object_clones":22,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":28,"num_read":9425,"num_read_kb":32113,"num_write":473,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.e","version":"148'549","reported_seq":6261,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.957911+0000","last_change":"2024-09-19T21:37:22.957911+0000","last_active":"2024-09-19T21:37:22.957911+0000","last_peered":"2024-09-19T21:37:22.957911+0000","last_clean":"2024-09-19T21:37:22.957911+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:37:22.957911+0000","last_undegraded":"2024-09-19T21:37:22.957911+0000","last_fullsized":"2024-09-19T21:37:22.957911+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061256,"stat_sum":{"num_bytes":30951424,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":8318,"num_read_kb":51766,"num_write":470,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.f","version":"139'774","reported_seq":4812,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.952265+0000","last_change":"2024-09-19T21:37:22.952849+0000","last_active":"2024-09-19T21:37:22.952265+0000","last_peered":"2024-09-19T21:37:22.952265+0000","last_clean":"2024-09-19T21:37:22.952265+0000","last_became_active":"2024-09-19T21:37:21.937190+0000","last_became_peered":"2024-09-19T21:37:21.937190+0000","last_unstale":"2024-09-19T21:37:22.952265+0000","last_undegraded":"2024-09-19T21:37:22.952265+0000","last_fullsized":"2024-09-19T21:37:22.952265+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:48:57.904336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001640416,"stat_sum":{"num_bytes":6682279,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":7191,"num_read_kb":14190,"num_write":854,"num_write_kb":4933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.10","version":"148'520","reported_seq":8423,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.186066+0000","last_change":"2024-09-19T21:37:22.972439+0000","last_active":"2024-09-19T21:37:23.186066+0000","last_peered":"2024-09-19T21:37:23.186066+0000","last_clean":"2024-09-19T21:37:23.186066+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:37:23.186066+0000","last_undegraded":"2024-09-19T21:37:23.186066+0000","last_fullsized":"2024-09-19T21:37:23.186066+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072386200000000003,"stat_sum":{"num_bytes":41775104,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":35,"num_read":9792,"num_read_kb":59244,"num_write":478,"num_write_kb":18042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.11","version":"148'373","reported_seq":3974,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.953944+0000","last_change":"2024-09-19T21:37:22.953944+0000","last_active":"2024-09-19T21:37:22.953944+0000","last_peered":"2024-09-19T21:37:22.953944+0000","last_clean":"2024-09-19T21:37:22.953944+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:37:22.953944+0000","last_undegraded":"2024-09-19T21:37:22.953944+0000","last_fullsized":"2024-09-19T21:37:22.953944+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027818959999999998,"stat_sum":{"num_bytes":50075172,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":4409,"num_read_kb":20492,"num_write":274,"num_write_kb":2915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.12","version":"148'260","reported_seq":7316,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:23.120105+0000","last_change":"2024-09-19T21:37:22.969223+0000","last_active":"2024-09-19T21:37:23.120105+0000","last_peered":"2024-09-19T21:37:23.120105+0000","last_clean":"2024-09-19T21:37:23.120105+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:37:23.120105+0000","last_undegraded":"2024-09-19T21:37:23.120105+0000","last_fullsized":"2024-09-19T21:37:23.120105+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0041663610000000004,"stat_sum":{"num_bytes":57249792,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":6850,"num_read_kb":72074,"num_write":274,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"5"}]},{"pgid":"3.13","version":"136'364","reported_seq":7719,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-19T21:37:22.972198+0000","last_change":"2024-09-19T21:37:22.972853+0000","last_active":"2024-09-19T21:37:22.972198+0000","last_peered":"2024-09-19T21:37:22.972198+0000","last_clean":"2024-09-19T21:37:22.972198+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:37:22.972198+0000","last_undegraded":"2024-09-19T21:37:22.972198+0000","last_fullsized":"2024-09-19T21:37:22.972198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095317700000000004,"stat_sum":{"num_bytes":34931328,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":8153,"num_read_kb":20757,"num_write":380,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1a","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1204843559,"num_objects":1568,"num_object_clones":439,"num_object_copies":3136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1568,"num_whiteouts":654,"num_read":297988,"num_read_kb":1055181,"num_write":17376,"num_write_kb":268907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4528,"num_bytes_recovered":277886826,"num_keys_recovered":645,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319721472,"data_stored":404875614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16217,"ondisk_log_size":16217,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102392,"kb_used_data":71268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94269448,"statfs":{"total":96636764160,"available":96531914752,"internally_reserved":0,"allocated":72978432,"data_stored":100965929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.92800000000000005}]}]},{"osd":4,"up_from":0,"seq":55834574890,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99812,"kb_used_data":50640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94272028,"statfs":{"total":96636764160,"available":96534556672,"internally_reserved":0,"allocated":51855360,"data_stored":59049489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122136,"kb_used_data":64776,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94249704,"statfs":{"total":96636764160,"available":96511696896,"internally_reserved":0,"allocated":66330624,"data_stored":84029262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.745}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.87},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.1020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114804,"kb_used_data":65632,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94257036,"statfs":{"total":96636764160,"available":96519204864,"internally_reserved":0,"allocated":67207168,"data_stored":84751814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.879}]}]},{"osd":3,"up_from":12,"seq":51539607593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100260,"kb_used_data":46996,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94271580,"statfs":{"total":96636764160,"available":96534097920,"internally_reserved":0,"allocated":48123904,"data_stored":58732893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.96899999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.622}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.93999999999999995}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.748},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.82499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":59731968,"data_stored":78320792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60575744,"data_stored":75900684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49938432,"data_stored":57997058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46391296,"data_stored":58015439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34279424,"data_stored":38636064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":68804608,"data_stored":96005577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:37:25.108 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [5, 3] 2024-09-19T21:37:25.108 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.4', '5', '3'] 2024-09-19T21:37:25.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.4 5 3 2024-09-19T21:37:25.123 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.123 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:25.172 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:25.166+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:25.244 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.244 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:25.297 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:25.294+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:25.373 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:25.366+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:25.411 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.411 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:25.474 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:25.470+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:25.528 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.529 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:25.581 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:25.578+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:25.676 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:25.670+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:25.691 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.691 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:25.777 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:25.770+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:25.811 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.811 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:25.878 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:25.870+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:25.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:25.978 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:25.979 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:25.974+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:25.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:25.997 INFO:teuthology.orchestra.run.smithi012.stderr:set 3.4 pg_upmap mapping to [5,3] 2024-09-19T21:37:26.071 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:26.079 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:26.074+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:26.091 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:26.092 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:26.096 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:26.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:26.124 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:26.160 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:26.180 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:26.174+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:26.197 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:26.232 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:26.235 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:26.250 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:26.250 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:26.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:26.281 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:26.278+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:26.382 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:26.378+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:26.412 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:26.412 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:26.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:26.482 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:26.478+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:26.533 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.910383794, 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:26.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:26.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.910362066, 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:26.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.91034133399999, 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.544 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:26.548 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.910306898, 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:26.552 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:26.553 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.910264101, 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.556 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.557 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:26.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90503859099999, 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.561 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:26.562 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.564 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.905016293, 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:26.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.904985438, 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:26.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.574 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.904942729, 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:26.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90491874200001, 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.580 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:26.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.583 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90488733799999, 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:26.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:26.589 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90486217199999, 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:26.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:26.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90481709399999, 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:26.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.598 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90479035, 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.599 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:26.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.603 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.90477705799999, 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:26.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:26.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:26.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 113.904760139, 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:26.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:26.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:26.612 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.504773366999999, 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:26.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:26.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.503206663999997, 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:26.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:26.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:26.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:26.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:26.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.625 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.503170384000001, 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:26.626 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:26.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:26.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 54.503150362, 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:26.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:26.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:26.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:26.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:26.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.640 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:26.641 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:26.641 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:26.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:26.643 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:26.638+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:26.743 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:26.738+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:26.758 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:26.758 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:26.758 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:26.758 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:26.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:26.844 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:26.838+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:26.888 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:26.889 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:26.889 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:26.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:26.946 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:26.942+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:26.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 191.07345284499999, 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:27.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:27.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:27.015 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753885514, 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:27.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:27.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753873748, 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:27.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753860855, 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:27.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.029 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753848832, 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:27.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.75383118800001, 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.75381539200001, 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:27.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753803541, 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:27.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:27.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:27.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753734697, 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:27.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.052 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753706557, 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:27.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.753683428, 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:27.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.738906808, 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:27.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:27.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:27.066 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.73157049300001, 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:27.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:27.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.731549603, 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:27.075 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.724474063, 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:27.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.72445351899999, 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:27.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:27.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:27.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.084 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.544099448, 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:27.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.544089021, 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.089 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:27.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.092 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.093 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.54407859200001, 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:27.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:27.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.544068529, 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:27.098 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:27.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:27.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.102 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:27.104 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:27.104 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:27.098+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:27.205 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:27.198+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:27.222 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:27.222 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:27.222 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:27.222 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:27.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:27.306 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:27.298+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:27.333 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:27.334 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:27.336 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:27.336 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:27.336 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:27.343 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:27.407 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:27.402+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:27.454 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 151.72185595100001, 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:27.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:27.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:27.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119900707, 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.461 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119454725, 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.466 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:27.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:27.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119340546, 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:27.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:27.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119324113, 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:27.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:27.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:27.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:27.479 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119308557, 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:27.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.481 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:27.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.11919448800001, 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.484 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:27.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119177525, 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:27.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119161541, 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:27.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:27.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:27.497 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119144366, 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.498 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:27.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.11912716800001, 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.503 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.507 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119110337, 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:27.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119093816, 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.512 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:27.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.11907121199999, 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.119054824, 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.521 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 101.118945943, 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:27.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:27.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.528 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:27.529 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 56.439914907999999, 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:27.530 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.531 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:27.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.533 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:27.534 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 56.439801758999998, 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.535 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.536 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:27.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.538 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:27.539 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 56.430536246000003, 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:27.540 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.541 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:27.542 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:27.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.544 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.544 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:27.544 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.545 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.546 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 56.430463821000004, 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:27.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.548 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:27.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:27.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:27.552 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:27.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:27.554 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:27.550+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:27.654 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:27.650+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:27.672 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:27.672 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:27.672 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:27.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:27.755 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:27.750+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:27.790 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:27.791 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:27.791 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:27.791 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:27.791 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:27.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:27.849 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:37:27.855 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:27.850+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.660113634, 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.66010009499999, 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.66007435500001, 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.660060141, 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.660039224, 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.66002384800001, 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.66000892, 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.6599563, 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65990672199999, 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.659889507, 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.659870369, 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.659855521, 2024-09-19T21:37:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65983252800001, 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.659100236, 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65904992599999, 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65898761400001, 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.657656981, 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.657521403, 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65744464400001, 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.65738592, 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.009 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:28.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:28.011 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:28.006+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:28.112 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:28.106+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:28.146 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:28.146 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:28.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:28.212 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:28.210+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:28.250 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:28.264 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:28.264 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:28.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:28.312 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:28.306+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.17942654399999, 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.17884663300001, 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.178783986, 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.178717666, 2024-09-19T21:37:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.178013988, 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177941166, 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177927428, 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177909739, 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177859353, 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.17783246400001, 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177791888, 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177770274, 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 120.177751075, 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 106.688433473, 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 106.68780999099999, 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 106.687477715, 2024-09-19T21:37:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.598386497999996, 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.575790142999999, 2024-09-19T21:37:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.575459475999999, 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 51.574939903999997, 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:28.477 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:28.478 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:28.478 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:28.478 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:28.478 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:28.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:28.479 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:28.474+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:28.493 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:28.516 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:28.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:28.579 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:28.574+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:28.634 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:28.634 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:28.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:28.689 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:28.686+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:28.757 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:28.757 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:28.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:28.780 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:28.774+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:28.876 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:28.876 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:28.881 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:28.874+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:28.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:28.982 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:28.978+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:29.014 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.014 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:29.082 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:29.078+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:29.139 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.139 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:29.183 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:29.178+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:29.253 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.253 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:29.284 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:29.278+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:29.372 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.373 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:29.387 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:29.382+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:29.487 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:29.482+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:29.491 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.491 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:29.590 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:29.586+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:29.655 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.655 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:29.690 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:29.686+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:29.776 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.776 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:29.792 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:29.786+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:29.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:37:29.893 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:29.886+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:29.941 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:29.942 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:29.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:29.996 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:29.990+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:30.094 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.090+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.116 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:30.116 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:30.127 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:30.196 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:30.190+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:30.251 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:30.251 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:30.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:30.296 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.290+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.369 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:30.369 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:30.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:30.397 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.394+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.487 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:30.487 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:30.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:30.498 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.494+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:30.598 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:30.594+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:30.610 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:30.611 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:30.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:30.684 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:30.699 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:30.694+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:30.707 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:30.729 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:30.752 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:30.773 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:30.786 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:30.787 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:30.787 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:30.787 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:30.787 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:30.795 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:30.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:30.797 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:30.798 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:30.799 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.798+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.900 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:30.898+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:30.904 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:30.904 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:30.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:30.904 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.701716674, 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:30.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.70113676299999, 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.701074116, 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:30.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.701007796, 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:30.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.70030411800001, 2024-09-19T21:37:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:30.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.700231296, 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:30.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.70021755800001, 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:30.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:30.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.700199869, 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:30.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.700149483, 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:30.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.70012259400001, 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.700082018, 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:30.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.700060404, 2024-09-19T21:37:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:30.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:30.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 122.70004120500001, 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:30.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.966 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.21072360300001, 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.210100121, 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 109.209767845, 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:30.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:30.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:30.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.120676627999998, 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.980 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.985 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.098080273000001, 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:30.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:30.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.097749606000001, 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:30.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.995 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 54.097230033999999, 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:30.996 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:30.998 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:30.999 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.000 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:31.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:31.002 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:30.998+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:31.016 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:31.016 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:31.016 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T21:37:31.016 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T21:37:31.017 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T21:37:31.017 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T21:37:31.027 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:37:31.102 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:31.098+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "" 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:31.126 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:31.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:31.203 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:31.198+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:31.243 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:31.243 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:31.243 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:31.243 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:31.243 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:31.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:31.303 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:31.298+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 155.617021607, 2024-09-19T21:37:31.370 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:31.372 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.373 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:31.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:31.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989682211, 2024-09-19T21:37:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:31.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989629745, 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.381 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.382 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.383 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.98959919799999, 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.386 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.387 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989576601, 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.390 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:31.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989554149, 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.396 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989539853, 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.401 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989523226, 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.404 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:31.408 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.98950459, 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:31.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:31.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989477925, 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989467714, 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.421 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.989455228, 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:31.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:31.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.98556311900001, 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.430 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:31.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.433 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:31.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:31.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.433 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.434 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.434 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.985525565, 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.435 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:31.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:31.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:31.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:31.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:31.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:31.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:31.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.444 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.98550772199999, 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:31.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:31.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:31.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:31.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:31.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.452 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.452 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:31.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:31.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:31.455 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.985495273, 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:31.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:31.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.460 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.98548458099999, 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.985473004, 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:31.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:31.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:31.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 110.56950104800001, 2024-09-19T21:37:31.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:31.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:31.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 22.222656894, 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:31.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:31.483 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:31.487 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:31.482+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:31.487 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:31.585 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:31.582+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:31.608 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:31.608 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:31.609 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:31.609 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:31.619 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:31.686 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:31.678+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:31.729 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:31.729 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:31.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:31.786 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:31.782+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:31.887 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:31.882+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.900 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.276976938, 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:31.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.27695521, 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.905 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:31.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.910 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.276934478, 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:31.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.27690004199999, 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.915 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:31.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.919 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.276857245, 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271631735, 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:31.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:31.928 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271609437, 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:31.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271578582, 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.934 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:31.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:31.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271535873, 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271511886, 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:31.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271480482, 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:31.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:31.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.952 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271455316, 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.953 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.956 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271410238, 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.957 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:31.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.27138349400001, 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.963 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271370202, 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:31.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.271353283, 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:31.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:31.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.871366510999998, 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:31.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:31.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:31.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.869799808000003, 2024-09-19T21:37:31.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:31.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.869763528, 2024-09-19T21:37:31.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:31.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:31.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:31.994 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 59.869743505999999, 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:31.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:31.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:31.999 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.000 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.000 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:32.002 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:31.998+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:32.102 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:32.094+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:32.124 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:32.124 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:32.125 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:32.205 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:32.198+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:32.255 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.255 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:32.256 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:32.256 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:32.304 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:32.298+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:32.375 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 196.438292174, 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.376 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:32.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.380 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:32.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:32.382 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:32.383 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.118724843, 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:32.384 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:32.385 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.386 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.387 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.118713077, 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:32.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.389 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:32.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:32.391 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.392 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.11870018400001, 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:32.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.394 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.395 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:32.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.11868816099999, 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:32.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:32.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:32.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:32.401 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.118670517, 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.402 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.403 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.404 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:32.405 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:32.406 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:32.407 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.118654721, 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.408 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.409 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:32.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.11864287, 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.413 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.414 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.415 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:32.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.118574026, 2024-09-19T21:37:32.417 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.419 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.420 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:32.421 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.11854588600001, 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.422 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.424 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.425 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.11852275699999, 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.427 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.429 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.430 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.103746137, 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:32.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:32.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.435 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.096409822, 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:32.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.437 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:32.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:32.440 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.096388932, 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:32.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.08931339199999, 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.445 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.447 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.449 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 120.089292848, 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:32.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.908938777, 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.457 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.459 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.90892835, 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:32.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:32.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.908917921, 2024-09-19T21:37:32.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:32.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:32.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.90890785800001, 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:32.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:32.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.472 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:32.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.474 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:32.476 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:32.474+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:32.577 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:32.574+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:32.636 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.636 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:32.636 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:32.636 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:32.677 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:32.674+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:32.754 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.755 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:32.755 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:32.755 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:32.755 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:32.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:32.778 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:32.774+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:32.813 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:32.879 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:32.878+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:32.900 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 157.16790360799999, 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.904 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:32.905 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56594836399999, 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56550238200001, 2024-09-19T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565388203, 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.916 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.917 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:32.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:32.920 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56537177, 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:32.921 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.922 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:32.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565356214, 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:32.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565242145, 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:32.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56522518200001, 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:32.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:32.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.938 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56520919800001, 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:32.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:32.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:32.943 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56519202299999, 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:32.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:32.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:32.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565174825, 2024-09-19T21:37:32.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:32.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:32.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565157994, 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:32.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565141473, 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.958 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:32.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565118869, 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:32.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.565102481, 2024-09-19T21:37:32.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:32.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:32.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 106.56499359999999, 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.885962565, 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:32.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.885849415999999, 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:32.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.987 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.876583902999997, 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:32.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:32.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:32.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 61.876511477999998, 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:32.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:32.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:32.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:33.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:33.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:33.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:33.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:33.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:33.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:33.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:33.001 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:33.001 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:33.001 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:33.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:33.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:33.004 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:33.004 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:33.002+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:33.025 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:33.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:37:33.050 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:33.089 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:33.103 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:33.102+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:33.116 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:33.120 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.120 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:33.204 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:33.202+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:33.234 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.234 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:33.305 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:33.302+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:33.354 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.355 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:33.406 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:33.402+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:33.475 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.475 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:33.506 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:33.506+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:33.598 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.598 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:33.608 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:33.602+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:33.708 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:33.702+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:33.713 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.713 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:33.810 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:33.806+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:33.832 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.833 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:33.910 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:33.910+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:33.954 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:33.955 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:33.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:34.012 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:34.006+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:34.051 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:37:34.075 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.076 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:34.113 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:34.110+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:34.214 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:34.210+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:34.238 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.238 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.250 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:34.315 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:34.310+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:34.360 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.360 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:34.416 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:34.410+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:34.478 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.479 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:34.517 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:34.514+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:34.597 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.597 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:34.655 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:34.654+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:34.715 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.715 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.718 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:34.718+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:34.725 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:34.819 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:34.814+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:34.831 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.831 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:34.920 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:34.914+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:34.946 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:34.946 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:34.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:35.020 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:35.014+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:35.068 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:35.068 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:35.079 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:35.121 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:35.118+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:35.135 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:35.189 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:35.189 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:35.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:35.221 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:35.218+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:35.286 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:35.322 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:35.318+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:35.326 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:35.326 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:35.327 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:35.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:35.348 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:35.370 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:35.393 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:35.395 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:35.396 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:35.422 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:35.418+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:35.447 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:35.447 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:35.447 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:35.448 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:35.448 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:35.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:35.524 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:35.518+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307610418, 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:35.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.568 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.569 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:35.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30759687899999, 2024-09-19T21:37:35.573 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:35.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.573 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.573 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.574 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:35.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.574 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.574 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:35.575 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.576 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307571139, 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:35.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:35.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307556925, 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:35.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.589 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307536008, 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:35.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30752063200001, 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30750570399999, 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:35.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.604 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307453084, 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30740350600001, 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:35.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307386291, 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:35.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:35.618 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.307367153, 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30735230499999, 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30732931200001, 2024-09-19T21:37:35.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:35.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30659702, 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:35.636 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30654670999999, 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.640 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30648439800001, 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:35.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:35.645 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.305153765, 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:35.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.305018187, 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30494142800001, 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.655 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:35.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 141.30488270399999, 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:35.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:35.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.663 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:35.664 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:35.664 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:35.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:35.666 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:35.662+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:35.766 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:35.762+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:35.781 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:35.782 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:35.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:35.867 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:35.866+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:35.944 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:35.945 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:35.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:35.968 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:35.966+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.86089547, 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.063 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.860315559, 2024-09-19T21:37:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.070 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.071 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:36.072 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.073 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.86025291199999, 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.074 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.077 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:36.078 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.86018659200001, 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.081 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.082 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:36.083 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:36.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.083 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859482914, 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.086 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.089 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:36.090 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:36.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.091 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.091 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.091 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859410092, 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.093 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.095 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.096 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.098 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859396354, 2024-09-19T21:37:36.099 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.85937866499999, 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:36.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:36.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:36.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:36.110 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859328279, 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:36.112 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.85930139, 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:36.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.120 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.121 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859260814, 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.124 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:36.127 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.8592392, 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:36.130 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:36.132 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 127.859220001, 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:36.133 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.134 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.135 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:36.136 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.369902399, 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.137 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.369278917, 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.142 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:36.143 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.144 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.145 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 114.36894664099999, 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.146 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.147 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.150 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.279855423999997, 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:36.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.257259069, 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:36.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.256928402, 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:36.161 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:36.164 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 59.256408829999998, 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:36.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.171 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:36.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:37:36.172 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:36.172 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:36.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:37:36.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:36.174 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:36.170+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:36.274 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:36.270+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:36.289 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:36.290 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:36.290 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:36.300 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:36.375 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:36.374+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:36.454 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:36.454 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:36.454 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:36.455 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:36.455 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:36.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:36.475 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:36.474+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:36.484 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:36.486 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:37:36.499 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-19T21:37:36.052975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":917180210,"num_objects":844,"num_object_clones":166,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":467,"num_read":300329,"num_read_kb":1057181,"num_write":18065,"num_write_kb":271428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4650,"num_bytes_recovered":284740131,"num_keys_recovered":645,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17434,"ondisk_log_size":17434,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":357772,"kb_used_data":103740,"kb_used_omap":7,"kb_used_meta":244088,"kb_avail":471501428,"statfs":{"total":483183820800,"available":482817462272,"internally_reserved":0,"allocated":106229760,"data_stored":100187945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":249946336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-177404163,"num_objects":-188,"num_object_clones":-54,"num_object_copies":-376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-188,"num_whiteouts":-39,"num_read":1177,"num_read_kb":1016,"num_write":233,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028376"},"pg_stats":[{"pgid":"3.1f","version":"157'492","reported_seq":4621,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.016136+0000","last_change":"2024-09-19T21:37:33.016639+0000","last_active":"2024-09-19T21:37:33.016136+0000","last_peered":"2024-09-19T21:37:33.016136+0000","last_clean":"2024-09-19T21:37:33.016136+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:37:33.016136+0000","last_undegraded":"2024-09-19T21:37:33.016136+0000","last_fullsized":"2024-09-19T21:37:33.016136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089988470000000008,"stat_sum":{"num_bytes":33867822,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":6349,"num_read_kb":38448,"num_write":432,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1e","version":"155'265","reported_seq":2573,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.244819+0000","last_change":"2024-09-19T21:37:31.030763+0000","last_active":"2024-09-19T21:37:31.244819+0000","last_peered":"2024-09-19T21:37:31.244819+0000","last_clean":"2024-09-19T21:37:31.244819+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:37:31.244819+0000","last_undegraded":"2024-09-19T21:37:31.244819+0000","last_fullsized":"2024-09-19T21:37:31.244819+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011233875000000001,"stat_sum":{"num_bytes":11823120,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":1784,"num_read_kb":7799,"num_write":196,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"3.1d","version":"157'1525","reported_seq":10383,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.848561+0000","last_change":"2024-09-19T21:37:32.849558+0000","last_active":"2024-09-19T21:37:32.848561+0000","last_peered":"2024-09-19T21:37:32.848561+0000","last_clean":"2024-09-19T21:37:32.848561+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:37:32.848561+0000","last_undegraded":"2024-09-19T21:37:32.848561+0000","last_fullsized":"2024-09-19T21:37:32.848561+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1525,"log_dups_size":0,"ondisk_log_size":1525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038552310000000002,"stat_sum":{"num_bytes":57513572,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":17247,"num_read_kb":39327,"num_write":1943,"num_write_kb":15206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1c","version":"158'603","reported_seq":16434,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.957589+0000","last_change":"2024-09-19T21:37:32.858072+0000","last_active":"2024-09-19T21:37:32.957589+0000","last_peered":"2024-09-19T21:37:32.957589+0000","last_clean":"2024-09-19T21:37:32.957589+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:37:32.957589+0000","last_undegraded":"2024-09-19T21:37:32.957589+0000","last_fullsized":"2024-09-19T21:37:32.957589+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012613961999999999,"stat_sum":{"num_bytes":5195300,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":17484,"num_read_kb":26616,"num_write":601,"num_write_kb":4144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1b","version":"156'394","reported_seq":4477,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.026861+0000","last_change":"2024-09-19T21:37:31.027204+0000","last_active":"2024-09-19T21:37:31.026861+0000","last_peered":"2024-09-19T21:37:31.026861+0000","last_clean":"2024-09-19T21:37:31.026861+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:37:31.026861+0000","last_undegraded":"2024-09-19T21:37:31.026861+0000","last_fullsized":"2024-09-19T21:37:31.026861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081283469999999993,"stat_sum":{"num_bytes":117397139,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":4144,"num_read_kb":25858,"num_write":358,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"3.1a","version":"157'1000","reported_seq":9105,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.056280+0000","last_change":"2024-09-19T21:37:32.855695+0000","last_active":"2024-09-19T21:37:33.056280+0000","last_peered":"2024-09-19T21:37:33.056280+0000","last_clean":"2024-09-19T21:37:33.056280+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:37:33.056280+0000","last_undegraded":"2024-09-19T21:37:33.056280+0000","last_fullsized":"2024-09-19T21:37:33.056280+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010491936,"stat_sum":{"num_bytes":10300963,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":11870,"num_read_kb":38157,"num_write":1467,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.19","version":"156'392","reported_seq":14720,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.022521+0000","last_change":"2024-09-19T21:37:31.022521+0000","last_active":"2024-09-19T21:37:31.022521+0000","last_peered":"2024-09-19T21:37:31.022521+0000","last_clean":"2024-09-19T21:37:31.022521+0000","last_became_active":"2024-09-19T21:37:15.907615+0000","last_became_peered":"2024-09-19T21:37:15.907615+0000","last_unstale":"2024-09-19T21:37:31.022521+0000","last_undegraded":"2024-09-19T21:37:31.022521+0000","last_fullsized":"2024-09-19T21:37:31.022521+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:01:19.735252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003443963,"stat_sum":{"num_bytes":18688146,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":14783,"num_read_kb":34061,"num_write":452,"num_write_kb":8769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"3.18","version":"156'304","reported_seq":10521,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.027691+0000","last_change":"2024-09-19T21:37:31.027691+0000","last_active":"2024-09-19T21:37:31.027691+0000","last_peered":"2024-09-19T21:37:31.027691+0000","last_clean":"2024-09-19T21:37:31.027691+0000","last_became_active":"2024-09-19T21:37:15.911405+0000","last_became_peered":"2024-09-19T21:37:15.911405+0000","last_unstale":"2024-09-19T21:37:31.027691+0000","last_undegraded":"2024-09-19T21:37:31.027691+0000","last_fullsized":"2024-09-19T21:37:31.027691+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:59:19.031902+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0090945850000000005,"stat_sum":{"num_bytes":23172654,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":10655,"num_read_kb":42216,"num_write":355,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"3.17","version":"158'318","reported_seq":6343,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.015504+0000","last_change":"2024-09-19T21:37:33.016031+0000","last_active":"2024-09-19T21:37:33.015504+0000","last_peered":"2024-09-19T21:37:33.015504+0000","last_clean":"2024-09-19T21:37:33.015504+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:37:33.015504+0000","last_undegraded":"2024-09-19T21:37:33.015504+0000","last_fullsized":"2024-09-19T21:37:33.015504+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085666760000000005,"stat_sum":{"num_bytes":2167296,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":6201,"num_read_kb":14803,"num_write":227,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.16","version":"158'364","reported_seq":11178,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.854789+0000","last_change":"2024-09-19T21:37:32.855169+0000","last_active":"2024-09-19T21:37:32.854789+0000","last_peered":"2024-09-19T21:37:32.854789+0000","last_clean":"2024-09-19T21:37:32.854789+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:37:32.854789+0000","last_undegraded":"2024-09-19T21:37:32.854789+0000","last_fullsized":"2024-09-19T21:37:32.854789+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0091204519999999994,"stat_sum":{"num_bytes":20295720,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":12237,"num_read_kb":30453,"num_write":283,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.15","version":"158'398","reported_seq":6930,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.857504+0000","last_change":"2024-09-19T21:37:32.857906+0000","last_active":"2024-09-19T21:37:32.857504+0000","last_peered":"2024-09-19T21:37:32.857504+0000","last_clean":"2024-09-19T21:37:32.857504+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:37:32.857504+0000","last_undegraded":"2024-09-19T21:37:32.857504+0000","last_fullsized":"2024-09-19T21:37:32.857504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048694810000000002,"stat_sum":{"num_bytes":2831563,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":6706,"num_read_kb":12121,"num_write":296,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.14","version":"158'346","reported_seq":10292,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.852321+0000","last_change":"2024-09-19T21:37:32.855331+0000","last_active":"2024-09-19T21:37:32.852321+0000","last_peered":"2024-09-19T21:37:32.852321+0000","last_clean":"2024-09-19T21:37:32.852321+0000","last_became_active":"2024-09-19T21:37:21.937346+0000","last_became_peered":"2024-09-19T21:37:21.937346+0000","last_unstale":"2024-09-19T21:37:32.852321+0000","last_undegraded":"2024-09-19T21:37:32.852321+0000","last_fullsized":"2024-09-19T21:37:32.852321+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:52:22.072608+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010491749999999999,"stat_sum":{"num_bytes":28349456,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":9807,"num_read_kb":27294,"num_write":278,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.029311+0000","last_change":"2024-09-19T21:37:18.848384+0000","last_active":"2024-09-19T21:37:32.029311+0000","last_peered":"2024-09-19T21:37:32.029311+0000","last_clean":"2024-09-19T21:37:32.029311+0000","last_became_active":"2024-09-19T21:37:18.847932+0000","last_became_peered":"2024-09-19T21:37:18.847932+0000","last_unstale":"2024-09-19T21:37:32.029311+0000","last_undegraded":"2024-09-19T21:37:32.029311+0000","last_fullsized":"2024-09-19T21:37:32.029311+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:29.428084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"156'522","reported_seq":8791,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.234191+0000","last_change":"2024-09-19T21:37:31.033770+0000","last_active":"2024-09-19T21:37:31.234191+0000","last_peered":"2024-09-19T21:37:31.234191+0000","last_clean":"2024-09-19T21:37:31.234191+0000","last_became_active":"2024-09-19T21:37:15.907862+0000","last_became_peered":"2024-09-19T21:37:15.907862+0000","last_unstale":"2024-09-19T21:37:31.234191+0000","last_undegraded":"2024-09-19T21:37:31.234191+0000","last_fullsized":"2024-09-19T21:37:31.234191+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.181084+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014458489,"stat_sum":{"num_bytes":47159885,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":8751,"num_read_kb":23520,"num_write":431,"num_write_kb":6477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.028654+0000","last_change":"2024-09-19T21:37:12.849531+0000","last_active":"2024-09-19T21:37:32.028654+0000","last_peered":"2024-09-19T21:37:32.028654+0000","last_clean":"2024-09-19T21:37:32.028654+0000","last_became_active":"2024-09-19T21:37:12.849150+0000","last_became_peered":"2024-09-19T21:37:12.849150+0000","last_unstale":"2024-09-19T21:37:32.028654+0000","last_undegraded":"2024-09-19T21:37:32.028654+0000","last_fullsized":"2024-09-19T21:37:32.028654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:39:56.298451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"156'727","reported_seq":8893,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.331719+0000","last_change":"2024-09-19T21:37:31.033117+0000","last_active":"2024-09-19T21:37:31.331719+0000","last_peered":"2024-09-19T21:37:31.331719+0000","last_clean":"2024-09-19T21:37:31.331719+0000","last_became_active":"2024-09-19T21:37:16.916361+0000","last_became_peered":"2024-09-19T21:37:16.916361+0000","last_unstale":"2024-09-19T21:37:31.331719+0000","last_undegraded":"2024-09-19T21:37:31.331719+0000","last_fullsized":"2024-09-19T21:37:31.331719+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:35:55.526552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013352892999999999,"stat_sum":{"num_bytes":18331678,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":10642,"num_read_kb":42131,"num_write":809,"num_write_kb":10719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.035597+0000","last_change":"2024-09-19T21:37:12.843070+0000","last_active":"2024-09-19T21:37:32.035597+0000","last_peered":"2024-09-19T21:37:32.035597+0000","last_clean":"2024-09-19T21:37:32.035597+0000","last_became_active":"2024-09-19T21:37:12.842703+0000","last_became_peered":"2024-09-19T21:37:12.842703+0000","last_unstale":"2024-09-19T21:37:32.035597+0000","last_undegraded":"2024-09-19T21:37:32.035597+0000","last_fullsized":"2024-09-19T21:37:32.035597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:10:55.086102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"158'515","reported_seq":13272,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.015924+0000","last_change":"2024-09-19T21:37:33.016219+0000","last_active":"2024-09-19T21:37:33.015924+0000","last_peered":"2024-09-19T21:37:33.015924+0000","last_clean":"2024-09-19T21:37:33.015924+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:37:33.015924+0000","last_undegraded":"2024-09-19T21:37:33.015924+0000","last_fullsized":"2024-09-19T21:37:33.015924+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0091001180000000008,"stat_sum":{"num_bytes":12464640,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":12540,"num_read_kb":41393,"num_write":384,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-19T21:37:29.997053+0000","last_change":"2024-09-19T21:37:06.857489+0000","last_active":"2024-09-19T21:37:29.997053+0000","last_peered":"2024-09-19T21:37:29.997053+0000","last_clean":"2024-09-19T21:37:29.997053+0000","last_became_active":"2024-09-19T21:37:06.857170+0000","last_became_peered":"2024-09-19T21:37:06.857170+0000","last_unstale":"2024-09-19T21:37:29.997053+0000","last_undegraded":"2024-09-19T21:37:29.997053+0000","last_fullsized":"2024-09-19T21:37:29.997053+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:21:16.051466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"158'855","reported_seq":5571,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.083329+0000","last_change":"2024-09-19T21:37:32.848416+0000","last_active":"2024-09-19T21:37:33.083329+0000","last_peered":"2024-09-19T21:37:33.083329+0000","last_clean":"2024-09-19T21:37:33.083329+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:37:33.083329+0000","last_undegraded":"2024-09-19T21:37:33.083329+0000","last_fullsized":"2024-09-19T21:37:33.083329+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029555800000000002,"stat_sum":{"num_bytes":32916040,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":8485,"num_read_kb":22636,"num_write":1062,"num_write_kb":8300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.3","version":"158'880","reported_seq":9581,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.087329+0000","last_change":"2024-09-19T21:37:32.855099+0000","last_active":"2024-09-19T21:37:33.087329+0000","last_peered":"2024-09-19T21:37:33.087329+0000","last_clean":"2024-09-19T21:37:33.087329+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:37:33.087329+0000","last_undegraded":"2024-09-19T21:37:33.087329+0000","last_fullsized":"2024-09-19T21:37:33.087329+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001479165,"stat_sum":{"num_bytes":1749078,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":13306,"num_read_kb":21186,"num_write":927,"num_write_kb":4941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.2","version":"21'2","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.027148+0000","last_change":"2024-09-19T21:37:12.849222+0000","last_active":"2024-09-19T21:37:32.027148+0000","last_peered":"2024-09-19T21:37:32.027148+0000","last_clean":"2024-09-19T21:37:32.027148+0000","last_became_active":"2024-09-19T21:37:12.846220+0000","last_became_peered":"2024-09-19T21:37:12.846220+0000","last_unstale":"2024-09-19T21:37:32.027148+0000","last_undegraded":"2024-09-19T21:37:32.027148+0000","last_fullsized":"2024-09-19T21:37:32.027148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:33:18.345371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"157'555","reported_seq":11981,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.016717+0000","last_change":"2024-09-19T21:37:33.017143+0000","last_active":"2024-09-19T21:37:33.016717+0000","last_peered":"2024-09-19T21:37:33.016717+0000","last_clean":"2024-09-19T21:37:33.016717+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:37:33.016717+0000","last_undegraded":"2024-09-19T21:37:33.016717+0000","last_fullsized":"2024-09-19T21:37:33.016717+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0079311109999999994,"stat_sum":{"num_bytes":21296708,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":16,"num_read":13277,"num_read_kb":50551,"num_write":512,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-19T21:37:29.997580+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:37:29.997580+0000","last_peered":"2024-09-19T21:37:29.997580+0000","last_clean":"2024-09-19T21:37:29.997580+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:37:29.997580+0000","last_undegraded":"2024-09-19T21:37:29.997580+0000","last_fullsized":"2024-09-19T21:37:29.997580+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"157'736","reported_seq":9040,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.845512+0000","last_change":"2024-09-19T21:37:32.846848+0000","last_active":"2024-09-19T21:37:32.845512+0000","last_peered":"2024-09-19T21:37:32.845512+0000","last_clean":"2024-09-19T21:37:32.845512+0000","last_became_active":"2024-09-19T21:37:15.910410+0000","last_became_peered":"2024-09-19T21:37:15.910410+0000","last_unstale":"2024-09-19T21:37:32.845512+0000","last_undegraded":"2024-09-19T21:37:32.845512+0000","last_fullsized":"2024-09-19T21:37:32.845512+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:01:14.472862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020778559999999999,"stat_sum":{"num_bytes":43412992,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":9776,"num_read_kb":35344,"num_write":968,"num_write_kb":10414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.025281+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:37:32.025281+0000","last_peered":"2024-09-19T21:37:32.025281+0000","last_clean":"2024-09-19T21:37:32.025281+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:37:32.025281+0000","last_undegraded":"2024-09-19T21:37:32.025281+0000","last_fullsized":"2024-09-19T21:37:32.025281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"156'441","reported_seq":12660,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.333372+0000","last_change":"2024-09-19T21:37:31.019930+0000","last_active":"2024-09-19T21:37:31.333372+0000","last_peered":"2024-09-19T21:37:31.333372+0000","last_clean":"2024-09-19T21:37:31.333372+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:37:31.333372+0000","last_undegraded":"2024-09-19T21:37:31.333372+0000","last_fullsized":"2024-09-19T21:37:31.333372+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013120880000000001,"stat_sum":{"num_bytes":74227871,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":13483,"num_read_kb":48556,"num_write":456,"num_write_kb":3499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"2.3","version":"19'1","reported_seq":179,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.035435+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:37:32.035435+0000","last_peered":"2024-09-19T21:37:32.035435+0000","last_clean":"2024-09-19T21:37:32.035435+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:37:32.035435+0000","last_undegraded":"2024-09-19T21:37:32.035435+0000","last_fullsized":"2024-09-19T21:37:32.035435+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":386,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.035219+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:37:32.035219+0000","last_peered":"2024-09-19T21:37:32.035219+0000","last_clean":"2024-09-19T21:37:32.035219+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:37:32.035219+0000","last_undegraded":"2024-09-19T21:37:32.035219+0000","last_fullsized":"2024-09-19T21:37:32.035219+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"157'494","reported_seq":15745,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.845372+0000","last_change":"2024-09-19T21:37:32.846655+0000","last_active":"2024-09-19T21:37:32.845372+0000","last_peered":"2024-09-19T21:37:32.845372+0000","last_clean":"2024-09-19T21:37:32.845372+0000","last_became_active":"2024-09-19T21:37:15.906980+0000","last_became_peered":"2024-09-19T21:37:15.906980+0000","last_unstale":"2024-09-19T21:37:32.845372+0000","last_undegraded":"2024-09-19T21:37:32.845372+0000","last_fullsized":"2024-09-19T21:37:32.845372+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:23:14.867134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001942292,"stat_sum":{"num_bytes":53842960,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":14278,"num_read_kb":62278,"num_write":542,"num_write_kb":13066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.9","version":"157'447","reported_seq":5735,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.852896+0000","last_change":"2024-09-19T21:37:32.853416+0000","last_active":"2024-09-19T21:37:32.852896+0000","last_peered":"2024-09-19T21:37:32.852896+0000","last_clean":"2024-09-19T21:37:32.852896+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:37:32.852896+0000","last_undegraded":"2024-09-19T21:37:32.852896+0000","last_fullsized":"2024-09-19T21:37:32.852896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071046399999999995,"stat_sum":{"num_bytes":51754496,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":6022,"num_read_kb":18872,"num_write":386,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.a","version":"158'614","reported_seq":8731,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.045857+0000","last_change":"2024-09-19T21:37:32.853752+0000","last_active":"2024-09-19T21:37:33.045857+0000","last_peered":"2024-09-19T21:37:33.045857+0000","last_clean":"2024-09-19T21:37:33.045857+0000","last_became_active":"2024-09-19T21:37:21.938850+0000","last_became_peered":"2024-09-19T21:37:21.938850+0000","last_unstale":"2024-09-19T21:37:33.045857+0000","last_undegraded":"2024-09-19T21:37:33.045857+0000","last_fullsized":"2024-09-19T21:37:33.045857+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:20.509636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087542000000000002,"stat_sum":{"num_bytes":11303002,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":18,"num_read":9831,"num_read_kb":47715,"num_write":681,"num_write_kb":13332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.b","version":"154'317","reported_seq":2497,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-19T21:37:31.027366+0000","last_change":"2024-09-19T21:37:31.027694+0000","last_active":"2024-09-19T21:37:31.027366+0000","last_peered":"2024-09-19T21:37:31.027366+0000","last_clean":"2024-09-19T21:37:31.027366+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:37:31.027366+0000","last_undegraded":"2024-09-19T21:37:31.027366+0000","last_fullsized":"2024-09-19T21:37:31.027366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0083762239999999998,"stat_sum":{"num_bytes":3990047,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":11,"num_read":2325,"num_read_kb":21249,"num_write":322,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"6"}]},{"pgid":"3.c","version":"157'313","reported_seq":3954,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.853916+0000","last_change":"2024-09-19T21:37:32.854353+0000","last_active":"2024-09-19T21:37:32.853916+0000","last_peered":"2024-09-19T21:37:32.853916+0000","last_clean":"2024-09-19T21:37:32.853916+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:37:32.853916+0000","last_undegraded":"2024-09-19T21:37:32.853916+0000","last_fullsized":"2024-09-19T21:37:32.853916+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001078283,"stat_sum":{"num_bytes":1681920,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":3740,"num_read_kb":13556,"num_write":247,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.d","version":"158'478","reported_seq":9595,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.854354+0000","last_change":"2024-09-19T21:37:32.854732+0000","last_active":"2024-09-19T21:37:32.854354+0000","last_peered":"2024-09-19T21:37:32.854354+0000","last_clean":"2024-09-19T21:37:32.854354+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:37:32.854354+0000","last_undegraded":"2024-09-19T21:37:32.854354+0000","last_fullsized":"2024-09-19T21:37:32.854354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0097303839999999999,"stat_sum":{"num_bytes":43665408,"num_objects":32,"num_object_clones":10,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":20,"num_read":9431,"num_read_kb":32116,"num_write":484,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.e","version":"157'577","reported_seq":6329,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.855387+0000","last_change":"2024-09-19T21:37:32.855725+0000","last_active":"2024-09-19T21:37:32.855387+0000","last_peered":"2024-09-19T21:37:32.855387+0000","last_clean":"2024-09-19T21:37:32.855387+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:37:32.855387+0000","last_undegraded":"2024-09-19T21:37:32.855387+0000","last_fullsized":"2024-09-19T21:37:32.855387+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0094358589999999996,"stat_sum":{"num_bytes":24332800,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":8318,"num_read_kb":51766,"num_write":478,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.f","version":"156'812","reported_seq":4962,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.855385+0000","last_change":"2024-09-19T21:37:32.856083+0000","last_active":"2024-09-19T21:37:32.855385+0000","last_peered":"2024-09-19T21:37:32.855385+0000","last_clean":"2024-09-19T21:37:32.855385+0000","last_became_active":"2024-09-19T21:37:21.937190+0000","last_became_peered":"2024-09-19T21:37:21.937190+0000","last_unstale":"2024-09-19T21:37:32.855385+0000","last_undegraded":"2024-09-19T21:37:32.855385+0000","last_fullsized":"2024-09-19T21:37:32.855385+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:48:57.904336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023206339999999998,"stat_sum":{"num_bytes":2217639,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":7364,"num_read_kb":14341,"num_write":873,"num_write_kb":4939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.10","version":"157'563","reported_seq":8579,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.855260+0000","last_change":"2024-09-19T21:37:32.856142+0000","last_active":"2024-09-19T21:37:32.855260+0000","last_peered":"2024-09-19T21:37:32.855260+0000","last_clean":"2024-09-19T21:37:32.855260+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:37:32.855260+0000","last_undegraded":"2024-09-19T21:37:32.855260+0000","last_fullsized":"2024-09-19T21:37:32.855260+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010995892,"stat_sum":{"num_bytes":15759360,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":9965,"num_read_kb":59395,"num_write":502,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.11","version":"157'395","reported_seq":4037,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.847416+0000","last_change":"2024-09-19T21:37:32.847777+0000","last_active":"2024-09-19T21:37:32.847416+0000","last_peered":"2024-09-19T21:37:32.847416+0000","last_clean":"2024-09-19T21:37:32.847416+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:37:32.847416+0000","last_undegraded":"2024-09-19T21:37:32.847416+0000","last_fullsized":"2024-09-19T21:37:32.847416+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002727227,"stat_sum":{"num_bytes":48013348,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":4409,"num_read_kb":20492,"num_write":282,"num_write_kb":2915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.12","version":"157'285","reported_seq":7383,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:33.016284+0000","last_change":"2024-09-19T21:37:33.016689+0000","last_active":"2024-09-19T21:37:33.016284+0000","last_peered":"2024-09-19T21:37:33.016284+0000","last_clean":"2024-09-19T21:37:33.016284+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:37:33.016284+0000","last_undegraded":"2024-09-19T21:37:33.016284+0000","last_fullsized":"2024-09-19T21:37:33.016284+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0078843119999999992,"stat_sum":{"num_bytes":53050368,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":6850,"num_read_kb":72074,"num_write":285,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.13","version":"158'387","reported_seq":7782,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T21:37:32.854403+0000","last_change":"2024-09-19T21:37:32.854790+0000","last_active":"2024-09-19T21:37:32.854403+0000","last_peered":"2024-09-19T21:37:32.854403+0000","last_clean":"2024-09-19T21:37:32.854403+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:37:32.854403+0000","last_undegraded":"2024-09-19T21:37:32.854403+0000","last_fullsized":"2024-09-19T21:37:32.854403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0098202800000000007,"stat_sum":{"num_bytes":23816832,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":8153,"num_read_kb":20757,"num_write":385,"num_write_kb":5733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":916589823,"num_objects":840,"num_object_clones":166,"num_object_copies":1680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":840,"num_whiteouts":467,"num_read":300213,"num_read_kb":1057081,"num_write":17904,"num_write_kb":269010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4637,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129331200,"data_stored":133194163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17314,"ondisk_log_size":17314,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56772,"kb_used_data":22236,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94315068,"statfs":{"total":96636764160,"available":96578629632,"internally_reserved":0,"allocated":22769664,"data_stored":21504758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.107},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0529999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.89600000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.86699999999999999}]}]},{"osd":4,"up_from":0,"seq":55834574892,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68716,"kb_used_data":17392,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303124,"statfs":{"total":96636764160,"available":96566398976,"internally_reserved":0,"allocated":17809408,"data_stored":16586417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.84899999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.78600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84624,"kb_used_data":24616,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94287216,"statfs":{"total":96636764160,"available":96550109184,"internally_reserved":0,"allocated":25206784,"data_stored":23852546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":0.748}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70036,"kb_used_data":20872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94301804,"statfs":{"total":96636764160,"available":96565047296,"internally_reserved":0,"allocated":21372928,"data_stored":20487432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.94699999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.85399999999999998}]}]},{"osd":3,"up_from":12,"seq":51539607595,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77624,"kb_used_data":18624,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94294216,"statfs":{"total":96636764160,"available":96557277184,"internally_reserved":0,"allocated":19070976,"data_stored":17756792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.749}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":1.079}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.94899999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0289999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19488768,"data_stored":19706364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22716416,"data_stored":22473198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15454208,"data_stored":15361221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17215488,"data_stored":16986567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34279424,"data_stored":38636064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20176896,"data_stored":20030749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:37:36.576 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 160.82314650500001, 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.578 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:36.579 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.580 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.581 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.583 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195807109, 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:36.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.588 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.589 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195754643, 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.590 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.591 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:36.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.19572409600001, 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.596 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:36.597 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.598 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.19570149899999, 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.599 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.600 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:36.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195679047, 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.605 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:36.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.608 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195664751, 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.609 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.611 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:36.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195648124, 2024-09-19T21:37:36.613 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.614 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:36.616 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.19562948799999, 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.620 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195602823, 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.623 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.625 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.626 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195592612, 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.628 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:36.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:36.630 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.195580126, 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:36.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.633 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.635 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.191688017, 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:36.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.639 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:36.640 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.191650463, 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.641 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.644 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.19163261999999, 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.646 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:36.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.191620171, 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.650 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:36.651 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:36.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.19160947899999, 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.655 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.656 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:36.658 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.659 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 116.191597902, 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.660 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.661 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:36.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:36.664 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 115.77562594600001, 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:36.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 27.428781791999999, 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:36.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:36.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:36.674 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:36.675 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:36.574+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:36.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:36.677 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:36.674+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:36.777 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:36.774+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:36.791 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:36.791 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:36.792 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:36.801 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:36.878 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:36.878+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:36.911 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:36.911 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:36.911 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:36.921 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:36.979 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:36.978+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.028 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.404781505, 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:37.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:37.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.404759777, 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:37.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:37.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.037 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.404739045, 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:37.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:37.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.042 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.40470460900001, 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:37.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.404661812, 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:37.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:37.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:37.051 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399436302, 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.052 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:37.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.056 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.39941400399999, 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:37.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399383149, 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:37.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.39934044, 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.066 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:37.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:37.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.070 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399316453, 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:37.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399285049, 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.075 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:37.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399259883, 2024-09-19T21:37:37.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:37.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.083 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399214805, 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399188061, 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:37.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:37.091 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.399174769, 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.092 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:37.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:37.096 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 124.39915784999999, 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:37.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.100 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.999171078000003, 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:37.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.105 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.997604374999995, 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:37.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:37.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:37.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:37.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.997568095000005, 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.113 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:37.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:37.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.119 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 64.997548073000004, 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:37.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:37.124 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.125 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.125 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.125 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.125 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:37.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:37.127 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:37:37.128 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:37.122+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:37.226 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:37.226+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:37.242 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_repop(client.4358.0:26018 3.19 e162/140 3:9952c55e:::rbd_object_map.1106504b696b:head v 162'417, mlcod=162'415)", 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:37:37.241865+0000", 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00059644899999999998, 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00065048799999999996, 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241865+0000", 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241865+0000", 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241863+0000", 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241880+0000", 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7229999999999999e-05 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241910+0000", 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9346e-05 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241922+0000", 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1803e-05 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241968+0000", 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6003e-05 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:37:37.241992+0000", 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4749e-05 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-09-19T21:37:37.246 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-09-19T21:37:37.247 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:37.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:37.328 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:37.322+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:37.409 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:37.414 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:37.414 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:37.415 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:37.415 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:37.425 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:37.466 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:37.466+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:37.529 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:37.526+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:37.561 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:37.583 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:37.584 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:37.584 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:37.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:37.605 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:37.630 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:37.626+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:37.643 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:37.669 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:37.705 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:37.705 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:37.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:37.731 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:37.726+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:37.832 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:37.826+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:37.876 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:37.876 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:37.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:37.933 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:37.930+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:38.053 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:38.050+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:38.126 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.126 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.134 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:38.130+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:38.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:38.235 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:38.234+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:38.239 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.239 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:38.336 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:38.330+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:38.369 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.369 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:38.437 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:38.434+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:38.490 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.490 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:38.545 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:38.542+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:38.606 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.607 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:38.639 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:38.634+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:38.722 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.722 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:38.740 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:38.738+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:38.842 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:38.838+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:38.887 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:38.887 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:38.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:38.943 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:38.942+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:39.002 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.002 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:39.044 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:39.042+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:39.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:37:39.129 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.129 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.138 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:39.145 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:39.138+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:39.246 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:39.242+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:39.291 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.291 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:39.347 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:39.342+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:39.407 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.408 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:39.448 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:39.442+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:39.525 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.525 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:39.549 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:39.546+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:39.644 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.644 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.650 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:39.646+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:39.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:39.706 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:39.750 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:39.746+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:39.807 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:39.807 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:39.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:39.844 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:39.851 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:39.850+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:39.864 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:39.884 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:39.908 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 203.98848668700001, 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668919356, 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.66890759, 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668894697, 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668882674, 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.66886503000001, 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.66884923400001, 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668837383, 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668768539, 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.668740399, 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.66871727, 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.65394065, 2024-09-19T21:37:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.64660433500001, 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.646583445, 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.639507905, 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.63948736099999, 2024-09-19T21:37:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.45913329, 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.459122863, 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.45911243400001, 2024-09-19T21:37:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 127.459102371, 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.017 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:40.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:40.020 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:40.020 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:40.020 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:40.020 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:40.021 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:40.021 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.014+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.104 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:37:40.119 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:40.118+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:40.183 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:40.184 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:40.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:40.220 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.214+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.310 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:40.310 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:40.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:40.321 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.314+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.421 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.418+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.430 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 164.69775192200001, 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095796678, 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095350696, 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095236517, 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095220084, 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095204528, 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.09509045900001, 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095073496, 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095057512, 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.095040337, 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.09502313900001, 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.09500630799999, 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.094989787, 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.09496718299999, 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.094950795, 2024-09-19T21:37:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 114.094841914, 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.415810879000006, 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.415697730000005, 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.406432217000003, 2024-09-19T21:37:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 69.406359792000003, 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:40.528 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:40.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:40.530 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.526+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.631 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:40.626+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:40.649 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:40.649 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:40.650 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:40.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:40.732 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:40.730+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:40.769 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:40.769 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:40.769 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:40.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:40.833 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:40.830+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62987147999999, 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.890 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:40.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.892 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62985794100001, 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.893 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.895 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629832201, 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:40.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:40.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629817987, 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62979707, 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629781694, 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:40.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62976676599999, 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629714146, 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62966456800001, 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629647353, 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:40.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629628215, 2024-09-19T21:37:40.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.942 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62961336699999, 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:40.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:40.946 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.629590374, 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.947 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:40.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:40.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62885808199999, 2024-09-19T21:37:40.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:40.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:40.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:40.957 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62880777199999, 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.960 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62874546, 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:40.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.965 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.968 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.627414827, 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.969 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:40.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:40.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.627279249, 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:40.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:40.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62720249, 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:40.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:40.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 146.62714376599999, 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:40.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:40.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:40.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:40.989 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:40.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:40.991 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:40.986+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:41.091 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:41.085+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:41.150 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:41.150 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:41.209 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:41.205+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:41.268 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:41.269 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:41.269 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:41.292 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:41.290+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18814211700001, 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.392 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:41.393 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.394 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.395 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.187562206, 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.399 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.187499559, 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:41.402 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.403 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.187433239, 2024-09-19T21:37:41.405 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:41.407 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.409 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.410 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18672956099999, 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.411 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.414 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.186656739, 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.416 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:41.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.419 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18664300099999, 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:41.422 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.423 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18662531199999, 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.425 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.426 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.427 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.428 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.429 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18657492599999, 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.433 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.436 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18654803699999, 2024-09-19T21:37:41.437 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.438 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.440 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18650746099999, 2024-09-19T21:37:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.444 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.446 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.186485847, 2024-09-19T21:37:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.448 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.449 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:41.450 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 133.18646664799999, 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.452 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.453 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:41.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:41.456 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 119.69714904600001, 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.457 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:41.458 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.459 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:41.460 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.461 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 119.696525564, 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:41.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.463 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:41.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.466 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 119.696193288, 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:41.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.468 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:41.469 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.470 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:41.471 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.607102071, 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:41.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.473 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:41.474 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:41.475 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:41.477 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.584505715999995, 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.478 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.480 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:41.481 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.482 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.584175048999995, 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:41.485 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.486 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:41.488 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 64.583655476999994, 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.493 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.495 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:41.495 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:41.489+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:41.503 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:41.503 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:41.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:37:41.596 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:41.589+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:41.610 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.610 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:41.610 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:41.610 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.620 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:41.696 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:41.689+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:41.772 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:41.772 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:41.772 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:41.772 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:41.797 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:41.794+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:41.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:37:41.829 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:37:41.842 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-19T21:37:41.089201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":621427987,"num_objects":740,"num_object_clones":147,"num_object_copies":1480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":740,"num_whiteouts":452,"num_read":301594,"num_read_kb":1058264,"num_write":18311,"num_write_kb":271487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4650,"num_bytes_recovered":284740131,"num_keys_recovered":645,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17748,"ondisk_log_size":17748,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":334424,"kb_used_data":86124,"kb_used_omap":7,"kb_used_meta":248184,"kb_avail":471524776,"statfs":{"total":483183820800,"available":482841370624,"internally_reserved":0,"allocated":88190976,"data_stored":82188944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7968,"internal_metadata":254140640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-170161994,"num_objects":-57,"num_object_clones":-16,"num_object_copies":-114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-57,"num_whiteouts":-13,"num_read":1050,"num_read_kb":907,"num_write":169,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027003"},"pg_stats":[{"pgid":"3.1f","version":"163'502","reported_seq":4661,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.990845+0000","last_change":"2024-09-19T21:37:37.852217+0000","last_active":"2024-09-19T21:37:37.990845+0000","last_peered":"2024-09-19T21:37:37.990845+0000","last_clean":"2024-09-19T21:37:37.990845+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:37:37.990845+0000","last_undegraded":"2024-09-19T21:37:37.990845+0000","last_fullsized":"2024-09-19T21:37:37.990845+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001410127,"stat_sum":{"num_bytes":33867792,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":6368,"num_read_kb":38462,"num_write":441,"num_write_kb":23196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.1e","version":"157'269","reported_seq":2603,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.057303+0000","last_change":"2024-09-19T21:37:36.057562+0000","last_active":"2024-09-19T21:37:36.057303+0000","last_peered":"2024-09-19T21:37:36.057303+0000","last_clean":"2024-09-19T21:37:36.057303+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:37:36.057303+0000","last_undegraded":"2024-09-19T21:37:36.057303+0000","last_fullsized":"2024-09-19T21:37:36.057303+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001241338,"stat_sum":{"num_bytes":10819584,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":1784,"num_read_kb":7799,"num_write":200,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"3.1d","version":"157'1525","reported_seq":10409,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.845935+0000","last_change":"2024-09-19T21:37:37.848140+0000","last_active":"2024-09-19T21:37:37.845935+0000","last_peered":"2024-09-19T21:37:37.845935+0000","last_clean":"2024-09-19T21:37:37.845935+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:37:37.845935+0000","last_undegraded":"2024-09-19T21:37:37.845935+0000","last_fullsized":"2024-09-19T21:37:37.845935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1525,"log_dups_size":0,"ondisk_log_size":1525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037161239999999999,"stat_sum":{"num_bytes":57513572,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":17247,"num_read_kb":39327,"num_write":1943,"num_write_kb":15206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.1c","version":"163'610","reported_seq":16486,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.060435+0000","last_change":"2024-09-19T21:37:37.855028+0000","last_active":"2024-09-19T21:37:38.060435+0000","last_peered":"2024-09-19T21:37:38.060435+0000","last_clean":"2024-09-19T21:37:38.060435+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:37:38.060435+0000","last_undegraded":"2024-09-19T21:37:38.060435+0000","last_fullsized":"2024-09-19T21:37:38.060435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054355500000000004,"stat_sum":{"num_bytes":5195300,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":17518,"num_read_kb":26648,"num_write":613,"num_write_kb":4145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.1b","version":"161'401","reported_seq":4510,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.057174+0000","last_change":"2024-09-19T21:37:36.057174+0000","last_active":"2024-09-19T21:37:36.057174+0000","last_peered":"2024-09-19T21:37:36.057174+0000","last_clean":"2024-09-19T21:37:36.057174+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:37:36.057174+0000","last_undegraded":"2024-09-19T21:37:36.057174+0000","last_fullsized":"2024-09-19T21:37:36.057174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015756209999999999,"stat_sum":{"num_bytes":62051475,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":4144,"num_read_kb":25858,"num_write":363,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"3.1a","version":"161'1001","reported_seq":9167,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.031023+0000","last_change":"2024-09-19T21:37:37.855379+0000","last_active":"2024-09-19T21:37:38.031023+0000","last_peered":"2024-09-19T21:37:38.031023+0000","last_clean":"2024-09-19T21:37:38.031023+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:37:38.031023+0000","last_undegraded":"2024-09-19T21:37:38.031023+0000","last_fullsized":"2024-09-19T21:37:38.031023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087554500000000003,"stat_sum":{"num_bytes":10300947,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":11906,"num_read_kb":38191,"num_write":1468,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.19","version":"161'408","reported_seq":14767,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.331603+0000","last_change":"2024-09-19T21:37:36.056604+0000","last_active":"2024-09-19T21:37:36.331603+0000","last_peered":"2024-09-19T21:37:36.331603+0000","last_clean":"2024-09-19T21:37:36.331603+0000","last_became_active":"2024-09-19T21:37:15.907615+0000","last_became_peered":"2024-09-19T21:37:15.907615+0000","last_unstale":"2024-09-19T21:37:36.331603+0000","last_undegraded":"2024-09-19T21:37:36.331603+0000","last_fullsized":"2024-09-19T21:37:36.331603+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:01:19.735252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00123507,"stat_sum":{"num_bytes":17823855,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":14820,"num_read_kb":34089,"num_write":468,"num_write_kb":8773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"3.18","version":"161'309","reported_seq":10562,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.301288+0000","last_change":"2024-09-19T21:37:36.057798+0000","last_active":"2024-09-19T21:37:36.301288+0000","last_peered":"2024-09-19T21:37:36.301288+0000","last_clean":"2024-09-19T21:37:36.301288+0000","last_became_active":"2024-09-19T21:37:15.911405+0000","last_became_peered":"2024-09-19T21:37:15.911405+0000","last_unstale":"2024-09-19T21:37:36.301288+0000","last_undegraded":"2024-09-19T21:37:36.301288+0000","last_fullsized":"2024-09-19T21:37:36.301288+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:59:19.031902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00146434,"stat_sum":{"num_bytes":11212334,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":10690,"num_read_kb":42246,"num_write":360,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"3.17","version":"158'318","reported_seq":6369,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.850842+0000","last_change":"2024-09-19T21:37:37.851690+0000","last_active":"2024-09-19T21:37:37.850842+0000","last_peered":"2024-09-19T21:37:37.850842+0000","last_clean":"2024-09-19T21:37:37.850842+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:37:37.850842+0000","last_undegraded":"2024-09-19T21:37:37.850842+0000","last_fullsized":"2024-09-19T21:37:37.850842+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002017001,"stat_sum":{"num_bytes":2167296,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":6201,"num_read_kb":14803,"num_write":227,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.16","version":"163'369","reported_seq":11209,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.852122+0000","last_change":"2024-09-19T21:37:37.852559+0000","last_active":"2024-09-19T21:37:37.852122+0000","last_peered":"2024-09-19T21:37:37.852122+0000","last_clean":"2024-09-19T21:37:37.852122+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:37:37.852122+0000","last_undegraded":"2024-09-19T21:37:37.852122+0000","last_fullsized":"2024-09-19T21:37:37.852122+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0068251270000000003,"stat_sum":{"num_bytes":1866280,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":12237,"num_read_kb":30453,"num_write":284,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.15","version":"162'401","reported_seq":6961,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.846621+0000","last_change":"2024-09-19T21:37:37.848391+0000","last_active":"2024-09-19T21:37:37.846621+0000","last_peered":"2024-09-19T21:37:37.846621+0000","last_clean":"2024-09-19T21:37:37.846621+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:37:37.846621+0000","last_undegraded":"2024-09-19T21:37:37.846621+0000","last_fullsized":"2024-09-19T21:37:37.846621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030731600000000001,"stat_sum":{"num_bytes":1310871,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":14,"num_read":6710,"num_read_kb":12123,"num_write":297,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.14","version":"162'350","reported_seq":10322,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.845548+0000","last_change":"2024-09-19T21:37:37.845851+0000","last_active":"2024-09-19T21:37:37.845548+0000","last_peered":"2024-09-19T21:37:37.845548+0000","last_clean":"2024-09-19T21:37:37.845548+0000","last_became_active":"2024-09-19T21:37:21.937346+0000","last_became_peered":"2024-09-19T21:37:21.937346+0000","last_unstale":"2024-09-19T21:37:37.845548+0000","last_undegraded":"2024-09-19T21:37:37.845548+0000","last_fullsized":"2024-09-19T21:37:37.845548+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:52:22.072608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001580653,"stat_sum":{"num_bytes":18199568,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":20,"num_read":9807,"num_read_kb":27294,"num_write":280,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.067123+0000","last_change":"2024-09-19T21:37:18.848384+0000","last_active":"2024-09-19T21:37:37.067123+0000","last_peered":"2024-09-19T21:37:37.067123+0000","last_clean":"2024-09-19T21:37:37.067123+0000","last_became_active":"2024-09-19T21:37:18.847932+0000","last_became_peered":"2024-09-19T21:37:18.847932+0000","last_unstale":"2024-09-19T21:37:37.067123+0000","last_undegraded":"2024-09-19T21:37:37.067123+0000","last_fullsized":"2024-09-19T21:37:37.067123+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:29.428084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"161'536","reported_seq":8833,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.057518+0000","last_change":"2024-09-19T21:37:36.057518+0000","last_active":"2024-09-19T21:37:36.057518+0000","last_peered":"2024-09-19T21:37:36.057518+0000","last_clean":"2024-09-19T21:37:36.057518+0000","last_became_active":"2024-09-19T21:37:15.907862+0000","last_became_peered":"2024-09-19T21:37:15.907862+0000","last_unstale":"2024-09-19T21:37:36.057518+0000","last_undegraded":"2024-09-19T21:37:36.057518+0000","last_fullsized":"2024-09-19T21:37:36.057518+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.181084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263676,"stat_sum":{"num_bytes":41410085,"num_objects":22,"num_object_clones":7,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":8755,"num_read_kb":23522,"num_write":438,"num_write_kb":6477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.066190+0000","last_change":"2024-09-19T21:37:12.849531+0000","last_active":"2024-09-19T21:37:37.066190+0000","last_peered":"2024-09-19T21:37:37.066190+0000","last_clean":"2024-09-19T21:37:37.066190+0000","last_became_active":"2024-09-19T21:37:12.849150+0000","last_became_peered":"2024-09-19T21:37:12.849150+0000","last_unstale":"2024-09-19T21:37:37.066190+0000","last_undegraded":"2024-09-19T21:37:37.066190+0000","last_fullsized":"2024-09-19T21:37:37.066190+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:39:56.298451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"161'749","reported_seq":8991,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.057972+0000","last_change":"2024-09-19T21:37:36.057972+0000","last_active":"2024-09-19T21:37:36.057972+0000","last_peered":"2024-09-19T21:37:36.057972+0000","last_clean":"2024-09-19T21:37:36.057972+0000","last_became_active":"2024-09-19T21:37:16.916361+0000","last_became_peered":"2024-09-19T21:37:16.916361+0000","last_unstale":"2024-09-19T21:37:36.057972+0000","last_undegraded":"2024-09-19T21:37:36.057972+0000","last_fullsized":"2024-09-19T21:37:36.057972+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:35:55.526552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001489165,"stat_sum":{"num_bytes":1900574,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":10742,"num_read_kb":42220,"num_write":824,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.080797+0000","last_change":"2024-09-19T21:37:12.843070+0000","last_active":"2024-09-19T21:37:37.080797+0000","last_peered":"2024-09-19T21:37:37.080797+0000","last_clean":"2024-09-19T21:37:37.080797+0000","last_became_active":"2024-09-19T21:37:12.842703+0000","last_became_peered":"2024-09-19T21:37:12.842703+0000","last_unstale":"2024-09-19T21:37:37.080797+0000","last_undegraded":"2024-09-19T21:37:37.080797+0000","last_fullsized":"2024-09-19T21:37:37.080797+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:10:55.086102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"163'520","reported_seq":13303,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.856770+0000","last_change":"2024-09-19T21:37:37.857055+0000","last_active":"2024-09-19T21:37:37.856770+0000","last_peered":"2024-09-19T21:37:37.856770+0000","last_clean":"2024-09-19T21:37:37.856770+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:37:37.856770+0000","last_undegraded":"2024-09-19T21:37:37.856770+0000","last_fullsized":"2024-09-19T21:37:37.856770+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072156099999999999,"stat_sum":{"num_bytes":11491328,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":12540,"num_read_kb":41393,"num_write":385,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-19T21:37:35.048226+0000","last_change":"2024-09-19T21:37:06.857489+0000","last_active":"2024-09-19T21:37:35.048226+0000","last_peered":"2024-09-19T21:37:35.048226+0000","last_clean":"2024-09-19T21:37:35.048226+0000","last_became_active":"2024-09-19T21:37:06.857170+0000","last_became_peered":"2024-09-19T21:37:06.857170+0000","last_unstale":"2024-09-19T21:37:35.048226+0000","last_undegraded":"2024-09-19T21:37:35.048226+0000","last_fullsized":"2024-09-19T21:37:35.048226+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:21:16.051466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"162'871","reported_seq":5615,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.845120+0000","last_change":"2024-09-19T21:37:37.845421+0000","last_active":"2024-09-19T21:37:37.845120+0000","last_peered":"2024-09-19T21:37:37.845120+0000","last_clean":"2024-09-19T21:37:37.845120+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:37:37.845120+0000","last_undegraded":"2024-09-19T21:37:37.845120+0000","last_fullsized":"2024-09-19T21:37:37.845120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013948960000000001,"stat_sum":{"num_bytes":16861216,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":8510,"num_read_kb":22656,"num_write":1076,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.3","version":"163'940","reported_seq":9821,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.013980+0000","last_change":"2024-09-19T21:37:37.850125+0000","last_active":"2024-09-19T21:37:38.013980+0000","last_peered":"2024-09-19T21:37:38.013980+0000","last_clean":"2024-09-19T21:37:38.013980+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:37:38.013980+0000","last_undegraded":"2024-09-19T21:37:38.013980+0000","last_fullsized":"2024-09-19T21:37:38.013980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046892440000000004,"stat_sum":{"num_bytes":1749078,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":13647,"num_read_kb":21489,"num_write":975,"num_write_kb":4963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"2.2","version":"21'2","reported_seq":196,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.065831+0000","last_change":"2024-09-19T21:37:12.849222+0000","last_active":"2024-09-19T21:37:37.065831+0000","last_peered":"2024-09-19T21:37:37.065831+0000","last_clean":"2024-09-19T21:37:37.065831+0000","last_became_active":"2024-09-19T21:37:12.846220+0000","last_became_peered":"2024-09-19T21:37:12.846220+0000","last_unstale":"2024-09-19T21:37:37.065831+0000","last_undegraded":"2024-09-19T21:37:37.065831+0000","last_fullsized":"2024-09-19T21:37:37.065831+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:33:18.345371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"163'559","reported_seq":12015,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.064813+0000","last_change":"2024-09-19T21:37:37.853546+0000","last_active":"2024-09-19T21:37:38.064813+0000","last_peered":"2024-09-19T21:37:38.064813+0000","last_clean":"2024-09-19T21:37:38.064813+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:37:38.064813+0000","last_undegraded":"2024-09-19T21:37:38.064813+0000","last_fullsized":"2024-09-19T21:37:38.064813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024744290000000002,"stat_sum":{"num_bytes":20838928,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":13284,"num_read_kb":50554,"num_write":514,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-19T21:37:35.048125+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:37:35.048125+0000","last_peered":"2024-09-19T21:37:35.048125+0000","last_clean":"2024-09-19T21:37:35.048125+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:37:35.048125+0000","last_undegraded":"2024-09-19T21:37:35.048125+0000","last_fullsized":"2024-09-19T21:37:35.048125+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"161'740","reported_seq":9070,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.844660+0000","last_change":"2024-09-19T21:37:37.845338+0000","last_active":"2024-09-19T21:37:37.844660+0000","last_peered":"2024-09-19T21:37:37.844660+0000","last_clean":"2024-09-19T21:37:37.844660+0000","last_became_active":"2024-09-19T21:37:15.910410+0000","last_became_peered":"2024-09-19T21:37:15.910410+0000","last_unstale":"2024-09-19T21:37:37.844660+0000","last_undegraded":"2024-09-19T21:37:37.844660+0000","last_fullsized":"2024-09-19T21:37:37.844660+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:01:14.472862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001447046,"stat_sum":{"num_bytes":1162240,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":9776,"num_read_kb":35344,"num_write":970,"num_write_kb":10414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.070192+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:37:37.070192+0000","last_peered":"2024-09-19T21:37:37.070192+0000","last_clean":"2024-09-19T21:37:37.070192+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:37:37.070192+0000","last_undegraded":"2024-09-19T21:37:37.070192+0000","last_fullsized":"2024-09-19T21:37:37.070192+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"161'464","reported_seq":12716,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.337060+0000","last_change":"2024-09-19T21:37:36.056773+0000","last_active":"2024-09-19T21:37:36.337060+0000","last_peered":"2024-09-19T21:37:36.337060+0000","last_clean":"2024-09-19T21:37:36.337060+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:37:36.337060+0000","last_undegraded":"2024-09-19T21:37:36.337060+0000","last_fullsized":"2024-09-19T21:37:36.337060+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011849250000000001,"stat_sum":{"num_bytes":41026613,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":13519,"num_read_kb":48582,"num_write":478,"num_write_kb":3503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"2.3","version":"19'1","reported_seq":184,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.080868+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:37:37.080868+0000","last_peered":"2024-09-19T21:37:37.080868+0000","last_clean":"2024-09-19T21:37:37.080868+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:37:37.080868+0000","last_undegraded":"2024-09-19T21:37:37.080868+0000","last_fullsized":"2024-09-19T21:37:37.080868+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":391,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.080629+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:37:37.080629+0000","last_peered":"2024-09-19T21:37:37.080629+0000","last_clean":"2024-09-19T21:37:37.080629+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:37:37.080629+0000","last_undegraded":"2024-09-19T21:37:37.080629+0000","last_fullsized":"2024-09-19T21:37:37.080629+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"162'498","reported_seq":15775,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.844718+0000","last_change":"2024-09-19T21:37:37.845040+0000","last_active":"2024-09-19T21:37:37.844718+0000","last_peered":"2024-09-19T21:37:37.844718+0000","last_clean":"2024-09-19T21:37:37.844718+0000","last_became_active":"2024-09-19T21:37:15.906980+0000","last_became_peered":"2024-09-19T21:37:15.906980+0000","last_unstale":"2024-09-19T21:37:37.844718+0000","last_undegraded":"2024-09-19T21:37:37.844718+0000","last_fullsized":"2024-09-19T21:37:37.844718+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:23:14.867134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010777289999999999,"stat_sum":{"num_bytes":46722064,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":19,"num_read":14278,"num_read_kb":62278,"num_write":544,"num_write_kb":13066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.9","version":"161'450","reported_seq":5764,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.845636+0000","last_change":"2024-09-19T21:37:37.846453+0000","last_active":"2024-09-19T21:37:37.845636+0000","last_peered":"2024-09-19T21:37:37.845636+0000","last_clean":"2024-09-19T21:37:37.845636+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:37:37.845636+0000","last_undegraded":"2024-09-19T21:37:37.845636+0000","last_fullsized":"2024-09-19T21:37:37.845636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022139490000000002,"stat_sum":{"num_bytes":12253696,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":6022,"num_read_kb":18872,"num_write":387,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.a","version":"163'644","reported_seq":8860,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.851206+0000","last_change":"2024-09-19T21:37:37.851206+0000","last_active":"2024-09-19T21:37:37.851206+0000","last_peered":"2024-09-19T21:37:37.851206+0000","last_clean":"2024-09-19T21:37:37.851206+0000","last_became_active":"2024-09-19T21:37:21.938850+0000","last_became_peered":"2024-09-19T21:37:21.938850+0000","last_unstale":"2024-09-19T21:37:37.851206+0000","last_undegraded":"2024-09-19T21:37:37.851206+0000","last_fullsized":"2024-09-19T21:37:37.851206+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:20.509636+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0063819829999999999,"stat_sum":{"num_bytes":4581414,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":10006,"num_read_kb":47866,"num_write":704,"num_write_kb":13338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.b","version":"160'320","reported_seq":2528,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-09-19T21:37:36.056860+0000","last_change":"2024-09-19T21:37:36.057201+0000","last_active":"2024-09-19T21:37:36.056860+0000","last_peered":"2024-09-19T21:37:36.056860+0000","last_clean":"2024-09-19T21:37:36.056860+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:37:36.056860+0000","last_undegraded":"2024-09-19T21:37:36.056860+0000","last_fullsized":"2024-09-19T21:37:36.056860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011902499999999999,"stat_sum":{"num_bytes":2955791,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":2328,"num_read_kb":21250,"num_write":325,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"f","length":"9"}]},{"pgid":"3.c","version":"157'313","reported_seq":3980,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.848721+0000","last_change":"2024-09-19T21:37:37.849546+0000","last_active":"2024-09-19T21:37:37.848721+0000","last_peered":"2024-09-19T21:37:37.848721+0000","last_clean":"2024-09-19T21:37:37.848721+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:37:37.848721+0000","last_undegraded":"2024-09-19T21:37:37.848721+0000","last_fullsized":"2024-09-19T21:37:37.848721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042340279999999999,"stat_sum":{"num_bytes":1681920,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":7,"num_read":3740,"num_read_kb":13556,"num_write":247,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.d","version":"160'480","reported_seq":9623,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.855904+0000","last_change":"2024-09-19T21:37:37.856260+0000","last_active":"2024-09-19T21:37:37.855904+0000","last_peered":"2024-09-19T21:37:37.855904+0000","last_clean":"2024-09-19T21:37:37.855904+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:37:37.855904+0000","last_undegraded":"2024-09-19T21:37:37.855904+0000","last_fullsized":"2024-09-19T21:37:37.855904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017697660000000001,"stat_sum":{"num_bytes":17975296,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":9431,"num_read_kb":32116,"num_write":484,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.e","version":"157'577","reported_seq":6355,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.848191+0000","last_change":"2024-09-19T21:37:37.848604+0000","last_active":"2024-09-19T21:37:37.848191+0000","last_peered":"2024-09-19T21:37:37.848191+0000","last_clean":"2024-09-19T21:37:37.848191+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:37:37.848191+0000","last_undegraded":"2024-09-19T21:37:37.848191+0000","last_fullsized":"2024-09-19T21:37:37.848191+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026062659999999999,"stat_sum":{"num_bytes":24332800,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":8318,"num_read_kb":51766,"num_write":478,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.f","version":"163'836","reported_seq":5085,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.021209+0000","last_change":"2024-09-19T21:37:37.853535+0000","last_active":"2024-09-19T21:37:38.021209+0000","last_peered":"2024-09-19T21:37:38.021209+0000","last_clean":"2024-09-19T21:37:38.021209+0000","last_became_active":"2024-09-19T21:37:21.937190+0000","last_became_peered":"2024-09-19T21:37:21.937190+0000","last_unstale":"2024-09-19T21:37:38.021209+0000","last_undegraded":"2024-09-19T21:37:38.021209+0000","last_fullsized":"2024-09-19T21:37:38.021209+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:48:57.904336+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0086188900000000006,"stat_sum":{"num_bytes":1037991,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":7535,"num_read_kb":14488,"num_write":890,"num_write_kb":4945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.10","version":"163'565","reported_seq":8607,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.860811+0000","last_change":"2024-09-19T21:37:37.861046+0000","last_active":"2024-09-19T21:37:37.860811+0000","last_peered":"2024-09-19T21:37:37.860811+0000","last_clean":"2024-09-19T21:37:37.860811+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:37:37.860811+0000","last_undegraded":"2024-09-19T21:37:37.860811+0000","last_fullsized":"2024-09-19T21:37:37.860811+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072950829999999999,"stat_sum":{"num_bytes":15647232,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":30,"num_read":9965,"num_read_kb":59395,"num_write":502,"num_write_kb":18048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.11","version":"163'418","reported_seq":4168,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:38.001878+0000","last_change":"2024-09-19T21:37:37.846265+0000","last_active":"2024-09-19T21:37:38.001878+0000","last_peered":"2024-09-19T21:37:38.001878+0000","last_clean":"2024-09-19T21:37:38.001878+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:37:38.001878+0000","last_undegraded":"2024-09-19T21:37:38.001878+0000","last_fullsized":"2024-09-19T21:37:38.001878+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002087874,"stat_sum":{"num_bytes":48013312,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":4613,"num_read_kb":20668,"num_write":300,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.12","version":"157'285","reported_seq":7409,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.850676+0000","last_change":"2024-09-19T21:37:37.851295+0000","last_active":"2024-09-19T21:37:37.850676+0000","last_peered":"2024-09-19T21:37:37.850676+0000","last_clean":"2024-09-19T21:37:37.850676+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:37:37.850676+0000","last_undegraded":"2024-09-19T21:37:37.850676+0000","last_fullsized":"2024-09-19T21:37:37.850676+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001713659,"stat_sum":{"num_bytes":53050368,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":6850,"num_read_kb":72074,"num_write":285,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"5"}]},{"pgid":"3.13","version":"162'400","reported_seq":7827,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-19T21:37:37.855449+0000","last_change":"2024-09-19T21:37:37.855852+0000","last_active":"2024-09-19T21:37:37.855449+0000","last_peered":"2024-09-19T21:37:37.855449+0000","last_clean":"2024-09-19T21:37:37.855449+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:37:37.855449+0000","last_undegraded":"2024-09-19T21:37:37.855449+0000","last_fullsized":"2024-09-19T21:37:37.855449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013509430000000001,"stat_sum":{"num_bytes":23816780,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":8187,"num_read_kb":20782,"num_write":400,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":620837600,"num_objects":736,"num_object_clones":147,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":452,"num_read":301478,"num_read_kb":1058164,"num_write":18150,"num_write_kb":269069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4637,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":111353856,"data_stored":115394033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17628,"ondisk_log_size":17628,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53220,"kb_used_data":22100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94318620,"statfs":{"total":96636764160,"available":96582266880,"internally_reserved":0,"allocated":22630400,"data_stored":21372434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.754},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.88800000000000001}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.92000000000000004}]}]},{"osd":4,"up_from":0,"seq":55834574893,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":13,"seq":55834574891,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63340,"kb_used_data":14172,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94308500,"statfs":{"total":96636764160,"available":96571904000,"internally_reserved":0,"allocated":14512128,"data_stored":13412699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80700,"kb_used_data":23328,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94291140,"statfs":{"total":96636764160,"available":96554127360,"internally_reserved":0,"allocated":23887872,"data_stored":22580876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.437}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.47199999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.014},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":0.627}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70800,"kb_used_data":17524,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94301040,"statfs":{"total":96636764160,"available":96564264960,"internally_reserved":0,"allocated":17944576,"data_stored":16883132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.315}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57099999999999995}]}]},{"osd":3,"up_from":12,"seq":51539607596,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66364,"kb_used_data":9000,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94305476,"statfs":{"total":96636764160,"available":96568807424,"internally_reserved":0,"allocated":9216000,"data_stored":7939803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.495}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.876},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.83699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15998976,"data_stored":16075088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21307392,"data_stored":21141282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12500992,"data_stored":12542478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7299072,"data_stored":7142602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":34279424,"data_stored":38636064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19968000,"data_stored":19856519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4809.0:1602 3.1f 3:fba488ec:::rbd_data.1106e5d314c7.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 11309056~54784,write 15736832~9216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 166.14235526499999, 2024-09-19T21:37:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.093515948000000002, 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4809", 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 1602 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748622+0000", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.748621+0000", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786506+0000", 2024-09-19T21:37:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.037885294 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786533+0000", 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7070999999999999e-05 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786545+0000", 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1351e-05 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.899 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786611+0000", 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6509999999999998e-05 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.786819+0000", 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000207581 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for subops from 1", 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.788378+0000", 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001559515 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "op_commit", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.832668+0000", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.044289664999999999 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842044+0000", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.0093759859999999993 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "commit_sent", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842107+0000", 2024-09-19T21:37:41.901 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2760000000000002e-05 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:34:55.842138+0000", 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1063999999999999e-05 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17095 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3515392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.902 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.515015869, 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06527637, 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17095 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.903 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375961+0000", 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375963+0000", 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375968+0000", 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8130000000000003e-06 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.375972+0000", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6449999999999998e-06 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441000+0000", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065028039999999995 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441110+0000", 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010964 2024-09-19T21:37:41.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441238+0000", 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128011 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17096 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3528704~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514963403, 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065507311999999998, 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17096 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376014+0000", 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.909 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376013+0000", 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376019+0000", 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8599999999999998e-06 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.910 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376030+0000", 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1796e-05 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376038+0000", 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.8790000000000003e-06 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.911 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441288+0000", 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065249527000000002 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441392+0000", 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104517 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441521+0000", 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000128739 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.912 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17097 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3542016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514932856, 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065774257000000003, 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17097 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376044+0000", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376048+0000", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376055+0000", 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0609999999999998e-06 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.915 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376062+0000", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7999999999999999e-06 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441565+0000", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065502801999999999 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441691+0000", 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000126215 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441818+0000", 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012710099999999999 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17098 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3555328~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514910259, 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066087663000000005, 2024-09-19T21:37:41.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17098 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.918 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376067+0000", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376066+0000", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376069+0000", 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.919 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376076+0000", 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9949999999999999e-06 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376082+0000", 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8170000000000004e-06 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.920 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441869+0000", 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065786552999999998 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.441944+0000", 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.5291999999999997e-05 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442154+0000", 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00021038 2024-09-19T21:37:41.921 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17099 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3568640~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51488780699999, 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066303678000000005, 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17099 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376089+0000", 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376091+0000", 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2069999999999998e-06 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376095+0000", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9040000000000002e-06 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376099+0000", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3340000000000003e-06 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442194+0000", 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066094578000000001 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442284+0000", 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9664000000000001e-05 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442393+0000", 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010928 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17100 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3581952~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514873511, 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066514659000000004, 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17100 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.927 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376104+0000", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376103+0000", 2024-09-19T21:37:41.928 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376106+0000", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2340000000000001e-06 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376111+0000", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.9989999999999999e-06 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376115+0000", 2024-09-19T21:37:41.929 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8960000000000002e-06 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442424+0000", 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066308288000000007 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442511+0000", 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6917999999999997e-05 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.930 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442618+0000", 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107602 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17101 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3595264~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514856884, 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066741624999999999, 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17101 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376120+0000", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376122+0000", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4210000000000002e-06 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376128+0000", 2024-09-19T21:37:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9970000000000004e-06 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376134+0000", 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4079999999999997e-06 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442667+0000", 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066533604999999996 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442753+0000", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5923999999999999e-05 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442862+0000", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000108662 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17102 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3608576~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:41.935 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514838248, 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067080631000000002, 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17102 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376139+0000", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376138+0000", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376141+0000", 2024-09-19T21:37:41.937 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.909e-06 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376154+0000", 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2621e-05 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.938 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376162+0000", 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1759999999999993e-06 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442900+0000", 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066737463999999996 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.442986+0000", 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6713999999999997e-05 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.939 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443219+0000", 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023312499999999999 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17103 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3621888~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.514811583, 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067273820999999998, 2024-09-19T21:37:41.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17103 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.941 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376166+0000", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376165+0000", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376167+0000", 2024-09-19T21:37:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0449999999999999e-06 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376171+0000", 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.506e-06 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376173+0000", 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.711e-06 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443251+0000", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067078034999999994 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443333+0000", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.1406e-05 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443439+0000", 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000106432 2024-09-19T21:37:41.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17104 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3635200~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51480137199999, 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067502224999999999, 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.945 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17104 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376176+0000", 2024-09-19T21:37:41.946 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376175+0000", 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376177+0000", 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.947 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376183+0000", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2700000000000004e-06 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376185+0000", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8229999999999999e-06 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443470+0000", 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067284990000000003 2024-09-19T21:37:41.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.949 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.949 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443561+0000", 2024-09-19T21:37:41.949 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0718999999999997e-05 2024-09-19T21:37:41.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443678+0000", 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000116854 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17105 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3648512~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51478888600001, 2024-09-19T21:37:41.950 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067730482999999994, 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17105 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376188+0000", 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.952 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376190+0000", 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7379999999999999e-06 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376193+0000", 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1870000000000001e-06 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.376195+0000", 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6029999999999999e-06 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443709+0000", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.067513134000000002 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443797+0000", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8802999999999997e-05 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.443919+0000", 2024-09-19T21:37:41.954 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012129 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17111 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3728384~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.510896777, 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065268962, 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.955 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17111 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380080+0000", 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.957 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380082+0000", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3800000000000001e-06 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380101+0000", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9048000000000001e-05 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380111+0000", 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3340000000000008e-06 2024-09-19T21:37:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445231+0000", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065120583999999995 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445271+0000", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0068999999999999e-05 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445349+0000", 2024-09-19T21:37:41.959 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7903000000000003e-05 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17112 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3741696~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.510859223, 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065393243000000004, 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17112 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380118+0000", 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380117+0000", 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380122+0000", 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3640000000000001e-06 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.962 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380129+0000", 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7819999999999994e-06 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380133+0000", 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3680000000000002e-06 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445373+0000", 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065240295000000004 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445429+0000", 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.6110000000000003e-05 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445511+0000", 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2037000000000001e-05 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17113 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3755008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51084138, 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065547255999999998, 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17113 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380136+0000", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380135+0000", 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380137+0000", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.9860000000000001e-06 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380143+0000", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3399999999999997e-06 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380146+0000", 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7889999999999999e-06 2024-09-19T21:37:41.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445540+0000", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065394684999999994 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445605+0000", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5147999999999994e-05 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445683+0000", 2024-09-19T21:37:41.968 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7582000000000004e-05 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17114 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3768320~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51082893100001, 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065813056999999994, 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17114 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380148+0000", 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380150+0000", 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.641e-06 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380154+0000", 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0300000000000004e-06 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380156+0000", 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.88e-06 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445714+0000", 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065557978000000003 2024-09-19T21:37:41.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445759+0000", 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4514000000000001e-05 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.445961+0000", 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000202268 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17115 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3781632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.510818239, 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066105965000000003, 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17115 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380159+0000", 2024-09-19T21:37:41.975 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380160+0000", 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.688e-06 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380163+0000", 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.819e-06 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.976 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380167+0000", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.523e-06 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446001+0000", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065834333999999994 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446095+0000", 2024-09-19T21:37:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4189000000000002e-05 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446265+0000", 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00016966299999999999 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:17116 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3794944~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.51080666199999, 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066249220999999997, 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 17116 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380170+0000", 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380172+0000", 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3630000000000001e-06 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380179+0000", 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9229999999999996e-06 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.981 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.380185+0000", 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3499999999999996e-06 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446287+0000", 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.066101966999999998 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446335+0000", 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.8804999999999998e-05 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.446420+0000", 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4165999999999993e-05 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:18909 3.0 3:0077a291:::rbd_data.1106b7b0d98d.0000000000000001:18 [read 3762176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 121.094834706, 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065200927000000006, 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 18909 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.984 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796142+0000", 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.985 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796145+0000", 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5280000000000002e-06 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796152+0000", 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7889999999999995e-06 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.796158+0000", 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3890000000000004e-06 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861191+0000", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.065033518999999998 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861249+0000", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8264999999999999e-05 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:40.861343+0000", 2024-09-19T21:37:41.987 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.3814000000000007e-05 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.4358.0:22843 3.1f 3:f8bac1f8:::rbd_object_map.11064e5a13b9.000000000000001a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 32.747990551999997, 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74790413899999997, 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:41.988 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.4358", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22843 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142987+0000", 2024-09-19T21:37:41.989 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.142983+0000", 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999967 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143004+0000", 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0545999999999999e-05 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143035+0000", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1633e-05 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143057+0000", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1702999999999999e-05 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143188+0000", 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013043700000000001 2024-09-19T21:37:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.143212+0000", 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4287e-05 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890196+0000", 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.74698385499999997 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.992 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890454+0000", 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000258359 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:37:09.890891+0000", 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000436556 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:41.993 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:41.995 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:41.989+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:41.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:42.021 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:42.096 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:42.089+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:42.117 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T21:37:42.141 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:42.141 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:42.141 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:42.141 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:42.152 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:42.179 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:42.197 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:42.193+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:42.199 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:42.219 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:42.246 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:42.262 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.262 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.271 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:42.275 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:42.293 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:42.294 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:42.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:42.298 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:42.293+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:42.379 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.379 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:42.401 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:42.398+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.492 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:42.502 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:42.498+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:42.602 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:42.602+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:42.607 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.607 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:42.703 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:42.697+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:42.729 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.729 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:42.804 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:42.802+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:42.849 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:42.849 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:42.856 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-09-19T21:37:42.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:42.904 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:42.897+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:43.006 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:43.002+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:43.040 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.040 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:43.106 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:43.106+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:43.152 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.152 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:43.209 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:43.205+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:43.279 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.280 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.290 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:43.307 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:43.306+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:43.400 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.400 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.408 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:43.401+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:43.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:43.509 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:43.501+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:43.517 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.517 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.527 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:43.609 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:43.605+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:43.680 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.680 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:43.710 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:43.706+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:43.798 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.799 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:43.810 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:43.806+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:43.911 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:43.905+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:43.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:44.012 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:44.010+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:44.034 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:44.034 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:44.043 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:44.113 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:44.110+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:44.160 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:44.160 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:44.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:44.217 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:44.213+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:44.278 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:44.278 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:44.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:44.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T21:37:44.334 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:44.334+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:44.417 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:44.413+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:44.424 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:44.424 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:44.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:44.483 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:44.509 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:44.518 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:44.513+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:44.528 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:44.535 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:44.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:44.536 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:44.536 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:44.536 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:44.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:44.551 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:44.575 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:44.598 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:44.600 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:44.619 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:44.613+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:44.658 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:44.658 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:44.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:44.658 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3782 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7512064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03545134500001, 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069062184999999998, 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3782 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.659 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620023+0000", 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620022+0000", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620026+0000", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9740000000000004e-06 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620033+0000", 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2660000000000004e-06 2024-09-19T21:37:44.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620040+0000", 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4139999999999999e-06 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688814+0000", 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068773964000000007 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.688880+0000", 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5833999999999993e-05 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689085+0000", 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020568499999999999 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3783 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7593984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:44.663 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03542961700001, 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069526736000000006, 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3783 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:44.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620045+0000", 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620044+0000", 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620047+0000", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8449999999999999e-06 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620054+0000", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.832e-06 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620060+0000", 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2029999999999997e-06 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689129+0000", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069068566999999997 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689269+0000", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000140698 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689571+0000", 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030201099999999998 2024-09-19T21:37:44.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3784 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7659520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03540888500001, 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069834235999999994, 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3784 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620065+0000", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620068+0000", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1760000000000002e-06 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620076+0000", 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.294e-06 2024-09-19T21:37:44.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620081+0000", 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689592+0000", 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069511242000000001 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689671+0000", 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9087999999999999e-05 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689900+0000", 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022825800000000001 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3785 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7741440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.672 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03537444899999, 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070112042999999999, 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3785 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620100+0000", 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620099+0000", 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620104+0000", 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2799999999999997e-06 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620122+0000", 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8176e-05 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620133+0000", 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1178e-05 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689931+0000", 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069797931999999993 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.689980+0000", 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8965000000000003e-05 2024-09-19T21:37:44.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690212+0000", 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023211600000000001 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3786 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7823360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.035331652, 2024-09-19T21:37:44.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070395077, 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3786 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620143+0000", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620142+0000", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620145+0000", 2024-09-19T21:37:44.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5390000000000002e-06 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620152+0000", 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9870000000000002e-06 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.620156+0000", 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6120000000000001e-06 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690246+0000", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070089735 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690302+0000", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6233999999999997e-05 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.690538+0000", 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023538000000000001 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.681 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3810 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 6979584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03010614199999, 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069705677999999993, 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3810 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625368+0000", 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625367+0000", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625372+0000", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2690000000000001e-06 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625379+0000", 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8709999999999997e-06 2024-09-19T21:37:44.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625385+0000", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.49e-06 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692564+0000", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067179461999999995 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.692620+0000", 2024-09-19T21:37:44.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5952000000000002e-05 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695074+0000", 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0024534959999999999 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3811 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7061504~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03008384399999, 2024-09-19T21:37:44.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070160274999999994, 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3811 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625391+0000", 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625390+0000", 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625393+0000", 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1520000000000001e-06 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625400+0000", 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8449999999999997e-06 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625405+0000", 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3009999999999997e-06 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695140+0000", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.069734620999999997 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695279+0000", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138765 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695551+0000", 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000271952 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.690 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3812 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7127040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03005298900001, 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070485416999999995, 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3812 2024-09-19T21:37:44.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625421+0000", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625424+0000", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7009999999999998e-06 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625432+0000", 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6680000000000001e-06 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625456+0000", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3493e-05 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695593+0000", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070137625999999995 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695664+0000", 2024-09-19T21:37:44.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1060000000000001e-05 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695907+0000", 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024247200000000001 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3813 3.a 3:55678db9:::rbd_data.1106f8c9a9e8.0000000000000001:16 [read 7208960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.03001028, 2024-09-19T21:37:44.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070879889000000001, 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3813 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625464+0000", 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625468+0000", 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.968e-06 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625476+0000", 2024-09-19T21:37:44.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1580000000000005e-06 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625483+0000", 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2130000000000002e-06 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.695967+0000", 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070484484999999999 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696060+0000", 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2638000000000003e-05 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696344+0000", 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028393800000000002 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3814 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:44.702 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02998629300001, 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070919182999999997, 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3814 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:44.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625488+0000", 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625491+0000", 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0539999999999999e-06 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625497+0000", 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5910000000000001e-06 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625503+0000", 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7509999999999997e-06 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696389+0000", 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070886433999999998 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696407+0000", 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7805e-05 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.706 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3815 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02995488900001, 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070933982000000007, 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3815 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625519+0000", 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625523+0000", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7799999999999998e-06 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625531+0000", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0209999999999999e-06 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625537+0000", 2024-09-19T21:37:44.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2060000000000004e-06 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696440+0000", 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903235999999994 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696453+0000", 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3302000000000001e-05 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3816 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02992972300001, 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070936065000000006, 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3816 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625545+0000", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625544+0000", 2024-09-19T21:37:44.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625547+0000", 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.83e-06 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625571+0000", 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4045000000000001e-05 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625583+0000", 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1554e-05 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696475+0000", 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070892000999999996 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696481+0000", 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0519999999999998e-06 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.714 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3817 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02988464500001, 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070930755999999998, 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3817 2024-09-19T21:37:44.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625590+0000", 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625589+0000", 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625593+0000", 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.924e-06 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625604+0000", 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1399e-05 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625611+0000", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3219999999999998e-06 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696510+0000", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070899604000000005 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696520+0000", 2024-09-19T21:37:44.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0203e-05 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3818 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02985790100001, 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070921656999999999, 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3818 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625616+0000", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625619+0000", 2024-09-19T21:37:44.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.649e-06 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625624+0000", 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1109999999999999e-06 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625627+0000", 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3450000000000002e-06 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696531+0000", 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070903337999999996 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696538+0000", 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5630000000000003e-06 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3819 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02984460900001, 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070950392000000001, 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3819 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625630+0000", 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625631+0000", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625637+0000", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0859999999999998e-06 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625641+0000", 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.253e-06 2024-09-19T21:37:44.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696569+0000", 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070927939999999995 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696580+0000", 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1511e-05 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3820 3.a 3.9db1e6aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.02982768999999, 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070951800999999995, 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3820 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625647+0000", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625646+0000", 2024-09-19T21:37:44.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625648+0000", 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8819999999999999e-06 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625655+0000", 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2319999999999998e-06 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.625658+0000", 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3160000000000001e-06 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696593+0000", 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.070935353000000007 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.696598+0000", 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2920000000000003e-06 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18148 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139809306611232 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.629840917999999, 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0680315309999999, 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18148 2024-09-19T21:37:44.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025633+0000", 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025632+0000", 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025640+0000", 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4120000000000001e-06 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025655+0000", 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4717e-05 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025667+0000", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2473999999999999e-05 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025713+0000", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5986999999999999e-05 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025719+0000", 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3439999999999997e-06 2024-09-19T21:37:44.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.087694+0000", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619744840000001 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.088032+0000", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033828599999999998 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093665+0000", 2024-09-19T21:37:44.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0056327169999999998 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.737 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.628274215000005, 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.066780939, 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027200+0000", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027199+0000", 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:44.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027206+0000", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2599999999999999e-06 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027218+0000", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1474999999999999e-05 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027228+0000", 2024-09-19T21:37:44.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.961e-06 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027386+0000", 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000157987 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027395+0000", 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8959999999999992e-06 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089315+0000", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0619201250000001 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089431+0000", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000116118 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.089694+0000", 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026285399999999997 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093392+0000", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003697858 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093740+0000", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00034807000000000002 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093825+0000", 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4838000000000003e-05 2024-09-19T21:37:44.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093981+0000", 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015653299999999999 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3:bc38be8a:::rbd_object_map.1106c2d222f1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.628237935000001, 2024-09-19T21:37:44.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0669020730000001, 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027236+0000", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027240+0000", 2024-09-19T21:37:44.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4750000000000004e-06 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027246+0000", 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0560000000000001e-06 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027252+0000", 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5079999999999996e-06 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027436+0000", 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018423499999999999 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027446+0000", 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0118999999999999e-05 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090747+0000", 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0633010030000001 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.090915+0000", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016815600000000001 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 2", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.091489+0000", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057329500000000003 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.093681+0000", 2024-09-19T21:37:44.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002192308 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094019+0000", 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033757699999999998 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094058+0000", 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9082999999999998e-05 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094138+0000", 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0780999999999998e-05 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4176 3.1d 3:ba5f96f5:::rbd_header.1106c2d222f1:head [watch reconnect cookie 139688107119120 gen 7] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 72.628217913, 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0671231910000001, 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:44.751 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4176 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:44.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027256+0000", 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027259+0000", 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6510000000000001e-06 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:44.753 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027265+0000", 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0290000000000002e-06 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027272+0000", 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0430000000000002e-06 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027471+0000", 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019869399999999999 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.754 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.027478+0000", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0979999999999996e-06 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092339+0000", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0648611100000001 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.092485+0000", 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014659800000000001 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:44.755 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:33.094380+0000", 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018943320000000001 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:44.756 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:44.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:44.758 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:44.754+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:44.858 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:44.853+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:44.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T21:37:44.940 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:44.940 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:44.940 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:44.940 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:44.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:44.959 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:44.953+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:45.051 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:45.059 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:45.058+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:45.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:45.161 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:45.158+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:45.198 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4163.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 209.26340208799999, 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.181665828, 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:45.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4163", 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.33:0/1360695049", 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932799+0000", 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932797+0000", 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932811+0000", 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3482000000000001e-05 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932830+0000", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9950999999999999e-05 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932850+0000", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9826e-05 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932981+0000", 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130295 2024-09-19T21:37:45.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:15.932999+0000", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7910999999999998e-05 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111500+0000", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.178501834 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.111825+0000", 2024-09-19T21:37:45.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032443000000000001 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 5", 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.112108+0000", 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028357600000000001 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.113666+0000", 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001557812 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114284+0000", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00061823200000000003 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114384+0000", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9562000000000007e-05 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:16.114465+0000", 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0867999999999999e-05 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2051 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 532480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94383475699999, 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065198712000000006, 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2051 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252366+0000", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252368+0000", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8530000000000001e-06 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252372+0000", 2024-09-19T21:37:45.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.726e-06 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252375+0000", 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7699999999999999e-06 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317300+0000", 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064924673000000002 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317430+0000", 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000130338 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317565+0000", 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134659 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2052 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 614400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:45.210 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94382299099999, 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065629705999999996, 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2052 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:45.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252378+0000", 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.212 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252380+0000", 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3029999999999998e-06 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252384+0000", 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9219999999999999e-06 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252388+0000", 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8909999999999997e-06 2024-09-19T21:37:45.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317602+0000", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065213642000000002 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.317702+0000", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100243 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318008+0000", 2024-09-19T21:37:45.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000306027 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2053 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.943810098, 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066037407000000006, 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.215 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2053 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252391+0000", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252393+0000", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9980000000000002e-06 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252397+0000", 2024-09-19T21:37:45.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9110000000000003e-06 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252400+0000", 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5310000000000002e-06 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318059+0000", 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065658823000000005 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318129+0000", 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0301000000000003e-05 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318428+0000", 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000299158 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2054 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.219 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94379807499999, 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066404490999999996, 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2054 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252403+0000", 2024-09-19T21:37:45.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252405+0000", 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.897e-06 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252414+0000", 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6600000000000007e-06 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252418+0000", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.625e-06 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318476+0000", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066058031000000003 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318552+0000", 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5718999999999998e-05 2024-09-19T21:37:45.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318808+0000", 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025587400000000001 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2055 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94378043099999, 2024-09-19T21:37:45.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066715953999999994, 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2055 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252421+0000", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252420+0000", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252425+0000", 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9490000000000004e-06 2024-09-19T21:37:45.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252430+0000", 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.211e-06 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252433+0000", 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9020000000000001e-06 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318853+0000", 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066419746000000002 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.318913+0000", 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9978000000000001e-05 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319137+0000", 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022347400000000001 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94376463500001, 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067065382000000007, 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2056 2024-09-19T21:37:45.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252436+0000", 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252438+0000", 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.013e-06 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252442+0000", 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9750000000000001e-06 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252445+0000", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319176+0000", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066730261999999999 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319242+0000", 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6068000000000005e-05 2024-09-19T21:37:45.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319502+0000", 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026007100000000001 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.943752784, 2024-09-19T21:37:45.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067328703000000004, 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2057 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252448+0000", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252450+0000", 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2359999999999999e-06 2024-09-19T21:37:45.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252455+0000", 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3309999999999997e-06 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252461+0000", 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0469999999999999e-06 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319519+0000", 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067058751999999999 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319573+0000", 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3653999999999998e-05 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319777+0000", 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020398599999999999 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1073152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94368394, 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067722832999999996, 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2058 2024-09-19T21:37:45.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252517+0000", 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252516+0000", 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252521+0000", 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9729999999999997e-06 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252531+0000", 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0122000000000001e-05 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252539+0000", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8250000000000005e-06 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319815+0000", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067275933999999996 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.319916+0000", 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010039600000000001 2024-09-19T21:37:45.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320240+0000", 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032444800000000002 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1155072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94365579999999, 2024-09-19T21:37:45.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068099739000000006, 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2059 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252545+0000", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252548+0000", 2024-09-19T21:37:45.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4960000000000001e-06 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252557+0000", 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1769999999999998e-06 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252563+0000", 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1909999999999997e-06 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320283+0000", 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.067720649999999993 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320386+0000", 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000102507 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320645+0000", 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000259189 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.246 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2060 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.94363267099999, 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068441528000000001, 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2060 2024-09-19T21:37:45.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252568+0000", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252571+0000", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0680000000000001e-06 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252579+0000", 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4390000000000003e-06 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.252585+0000", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6730000000000003e-06 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320690+0000", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068105272999999994 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.320746+0000", 2024-09-19T21:37:45.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5541000000000002e-05 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.321010+0000", 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026395800000000001 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2109 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2555904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.928856051, 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066519957000000005, 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2109 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:45.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267345+0000", 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267348+0000", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0699999999999998e-06 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267359+0000", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.267370+0000", 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0539e-05 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331183+0000", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.063813255999999999 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.331290+0000", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000106434 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.333865+0000", 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025754369999999999 2024-09-19T21:37:45.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2146 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.92151973599999, 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065243427000000007, 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.256 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2146 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274681+0000", 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274684+0000", 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.185e-06 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274691+0000", 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4350000000000002e-06 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274696+0000", 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0710000000000002e-06 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339614+0000", 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064918355999999997 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339677+0000", 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3312000000000006e-05 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339925+0000", 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024739400000000001 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2147 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2564096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.92149884599999, 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065529731999999993, 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2147 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274702+0000", 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274705+0000", 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2969999999999999e-06 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274713+0000", 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9200000000000004e-06 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.274719+0000", 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8590000000000001e-06 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.339960+0000", 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065241384999999999 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340014+0000", 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3238000000000001e-05 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.340232+0000", 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021846600000000001 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2182 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.265 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.914423306, 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064934139000000002, 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2182 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281778+0000", 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281777+0000", 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281781+0000", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2990000000000001e-06 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281787+0000", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7129999999999999e-06 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281793+0000", 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8379999999999998e-06 2024-09-19T21:37:45.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346422+0000", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064628496999999993 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346492+0000", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0108e-05 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346712+0000", 2024-09-19T21:37:45.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022009699999999999 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:2183 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 2572288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.91440276200001, 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065229423999999994, 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.270 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 2183 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281798+0000", 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281801+0000", 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0359999999999998e-06 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.272 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281808+0000", 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.281816+0000", 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4509999999999992e-06 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346765+0000", 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064948318000000005 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.346816+0000", 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1504e-05 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.347028+0000", 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211657 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.274 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3056 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.734048691, 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064962226999999997, 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3056 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462152+0000", 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462154+0000", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0669999999999999e-06 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462157+0000", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2449999999999998e-06 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462160+0000", 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.993e-06 2024-09-19T21:37:45.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526664+0000", 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.064503961999999998 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.526775+0000", 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110227 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527115+0000", 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033998399999999999 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3057 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5062656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.73403826399999, 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065662779000000004, 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.279 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3057 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:45.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462163+0000", 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462164+0000", 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7999999999999999e-06 2024-09-19T21:37:45.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462168+0000", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.304e-06 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462171+0000", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.056e-06 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527226+0000", 2024-09-19T21:37:45.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065055532999999999 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527365+0000", 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013826499999999999 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527826+0000", 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046108999999999999 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.283 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3058 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5144576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.73402783500001, 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066118178, 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3058 2024-09-19T21:37:45.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462173+0000", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462175+0000", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6130000000000001e-06 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462178+0000", 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4130000000000002e-06 2024-09-19T21:37:45.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462181+0000", 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.033e-06 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527882+0000", 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.065700578999999995 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.527975+0000", 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3627000000000003e-05 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528291+0000", 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031617199999999998 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4964.0:3059 3.10 3:0ee46a8a:::rbd_data.1106f8c9a9e8.0000000000000000:16 [read 5226496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:45.288 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 132.73401777199999, 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066322144, 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4964", 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/1625744326", 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 3059 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:45.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462183+0000", 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462185+0000", 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7290000000000001e-06 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462188+0000", 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3110000000000001e-06 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.462191+0000", 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.128e-06 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528342+0000", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.066150865000000003 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528412+0000", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9793000000000001e-05 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:32.528505+0000", 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3579000000000006e-05 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.293 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.293 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:45.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:45.295 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:45.289+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:45.395 INFO:tasks.ceph.osd.1.smithi012.stderr:2024-09-19T21:37:45.394+0000 7fb49cf9a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 20944) UID: 0 2024-09-19T21:37:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:45.413 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-09-19T21:37:45.413 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:45.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:45.497 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:45.489+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:45.530 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-09-19T21:37:45.530 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-09-19T21:37:45.531 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-09-19T21:37:45.531 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:45.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:45.597 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:45.593+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4809.0:1592 3.1d 3:bd8d7df6:::rbd_data.1106e5d314c7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19312640~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 169.91742967900001, 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056394075000000002, 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4809", 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/524559831", 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 1592 2024-09-19T21:37:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729199+0000", 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.729198+0000", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750216+0000", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.021017662999999999 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750248+0000", 2024-09-19T21:37:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2097000000000001e-05 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750264+0000", 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6079e-05 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750336+0000", 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2107000000000006e-05 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.750522+0000", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018573900000000001 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for subops from 0", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.751596+0000", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010748419999999999 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op_commit", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.781462+0000", 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029865583000000001 2024-09-19T21:37:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_commit_rec", 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785366+0000", 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039042339999999999 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785505+0000", 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138858 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:34:55.785593+0000", 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8111999999999999e-05 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5046 3.16 3:6bfaaadd:::rbd_header.11064e5a13b9:head [watch reconnect cookie 140195181682720 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.315474435, 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058609139999999997, 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5046 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331154+0000", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331157+0000", 2024-09-19T21:37:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9349999999999999e-06 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331163+0000", 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8459999999999996e-06 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331167+0000", 2024-09-19T21:37:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4059999999999998e-06 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383497+0000", 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.052329572999999997 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.383571+0000", 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4721999999999994e-05 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389763+0000", 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.006192197 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5149 3.16 3:699b2196:::rbd_data.1106f26f138b.0000000000000016:19 [read 206848~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.315028453, 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055852010000000001, 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5149 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331600+0000", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331603+0000", 2024-09-19T21:37:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5009999999999999e-06 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331608+0000", 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5269999999999998e-06 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331613+0000", 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7269999999999998e-06 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386613+0000", 2024-09-19T21:37:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055000642000000002 2024-09-19T21:37:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.386692+0000", 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8944999999999997e-05 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387452+0000", 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00075998700000000005 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5204 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314914274, 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055740522000000001, 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5204 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331714+0000", 2024-09-19T21:37:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331717+0000", 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.694e-06 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331722+0000", 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7379999999999999e-06 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331727+0000", 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7119999999999999e-06 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387232+0000", 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055504998999999999 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387276+0000", 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3958000000000002e-05 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387455+0000", 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017886700000000001 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5205 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314897841, 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055856588999999998, 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5205 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331731+0000", 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331733+0000", 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2869999999999999e-06 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331738+0000", 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4809999999999997e-06 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331743+0000", 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7589999999999997e-06 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387475+0000", 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055731663000000001 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387522+0000", 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7376999999999997e-05 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387588+0000", 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5313000000000003e-05 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5206 3.e 3:753fe1b9:::rbd_data.1106f26f138b.0000000000000022:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314882285, 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055984697999999999, 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5206 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331746+0000", 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331757+0000", 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0747999999999999e-05 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331764+0000", 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331768+0000", 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6550000000000003e-06 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387620+0000", 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055851030000000003 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387686+0000", 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6179000000000005e-05 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387731+0000", 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5479000000000003e-05 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5222 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 4096~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314768216, 2024-09-19T21:37:45.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969974999999999, 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5222 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331861+0000", 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331860+0000", 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.687 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331863+0000", 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3719999999999999e-06 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331868+0000", 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2610000000000001e-06 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331873+0000", 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8319999999999996e-06 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387610+0000", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.055736814000000003 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.387668+0000", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8145e-05 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388831+0000", 2024-09-19T21:37:45.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011628789999999999 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5223 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314751253, 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057227289000000001, 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.690 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5223 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331878+0000", 2024-09-19T21:37:45.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331877+0000", 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331880+0000", 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5569999999999998e-06 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331885+0000", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4029999999999998e-06 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331890+0000", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5410000000000002e-06 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388860+0000", 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.056969872999999997 2024-09-19T21:37:45.693 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.388910+0000", 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0781000000000001e-05 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389105+0000", 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019451399999999999 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5224 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 125952~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314735269, 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057472328000000003, 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5224 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.695 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331894+0000", 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.696 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331893+0000", 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331896+0000", 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3429999999999998e-06 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331902+0000", 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5200000000000003e-06 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.697 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331907+0000", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9760000000000003e-06 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389120+0000", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057213473000000001 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389169+0000", 2024-09-19T21:37:45.698 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8534e-05 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389366+0000", 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019681599999999999 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5225 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 186368~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.699 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314718094, 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057758087, 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5225 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.700 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331911+0000", 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331910+0000", 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.701 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331913+0000", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.452e-06 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331919+0000", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9340000000000003e-06 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331924+0000", 2024-09-19T21:37:45.702 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8219999999999997e-06 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389401+0000", 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057477192000000003 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389469+0000", 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8067999999999999e-05 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.703 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389669+0000", 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019991599999999999 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5226 3.16 3:6bf3fd16:::rbd_data.1106f26f138b.0000000000000026:19 [read 246784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.31470089600001, 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058041874, 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5226 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.705 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331928+0000", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331927+0000", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331930+0000", 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-09-19T21:37:45.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331936+0000", 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9499999999999998e-06 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331941+0000", 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6750000000000001e-06 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.707 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389821+0000", 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.057879939999999998 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389896+0000", 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4975999999999995e-05 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389970+0000", 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3998000000000005e-05 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.708 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5231 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 25600~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.31468406499999, 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058030448999999998, 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5231 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331945+0000", 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.710 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331944+0000", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331947+0000", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2000000000000001e-06 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331952+0000", 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6640000000000003e-06 2024-09-19T21:37:45.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331957+0000", 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8439999999999997e-06 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389803+0000", 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0578462 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.712 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389893+0000", 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9448000000000001e-05 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.389975+0000", 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2403000000000004e-05 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5232 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 86016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.713 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314667544, 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058217583000000003, 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5232 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.714 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331961+0000", 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.715 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331969+0000", 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0090000000000007e-06 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331975+0000", 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0979999999999999e-06 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.716 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331980+0000", 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9010000000000002e-06 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390000+0000", 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058020631000000003 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390090+0000", 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9170000000000002e-05 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.717 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390179+0000", 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9214000000000003e-05 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5233 3.e 3:7606e668:::rbd_data.1106f26f138b.0000000000000035:19 [read 146432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.31464493999999, 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058411670999999998, 2024-09-19T21:37:45.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5233 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.719 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331984+0000", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331983+0000", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331986+0000", 2024-09-19T21:37:45.720 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.751e-06 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331992+0000", 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6869999999999999e-06 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.331996+0000", 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6020000000000002e-06 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.721 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390222+0000", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058225899999999997 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390318+0000", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5353999999999995e-05 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390396+0000", 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7892000000000006e-05 2024-09-19T21:37:45.722 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5234 3.e 3.1667606e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314628552, 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058504941999999997, 2024-09-19T21:37:45.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5234 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332000+0000", 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.725 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332002+0000", 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.368e-06 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332008+0000", 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9200000000000001e-06 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332013+0000", 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5709999999999999e-06 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.726 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390496+0000", 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058483578000000001 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390505+0000", 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8249999999999994e-06 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.727 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5332.0:5279 3.16 3.bb555fd6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 119.314519671, 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058342475999999997, 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5332", 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/262323201", 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 5279 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.728 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332109+0000", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332108+0000", 2024-09-19T21:37:45.729 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332112+0000", 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.014e-06 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332119+0000", 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6529999999999997e-06 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.730 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.332124+0000", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.3120000000000001e-06 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390444+0000", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.058319201000000001 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:35:46.390452+0000", 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9160000000000001e-06 2024-09-19T21:37:45.731 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4358.0:18147 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 74.635488636000005, 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139681570000001, 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4358", 2024-09-19T21:37:45.732 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/3273116364", 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 18147 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011140+0000", 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011139+0000", 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.999999 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011146+0000", 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0040000000000002e-06 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.734 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011161+0000", 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5512e-05 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011171+0000", 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7100000000000002e-06 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023687+0000", 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012516073000000001 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.735 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023698+0000", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0292000000000001e-05 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025090+0000", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0013920599999999 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025108+0000", 2024-09-19T21:37:45.736 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8689999999999999e-05 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4173 3.1d 3.af69fa5d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 74.635375487000005, 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.013913096, 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.737 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4173 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011253+0000", 2024-09-19T21:37:45.738 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011252+0000", 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011257+0000", 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4450000000000004e-06 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.739 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011265+0000", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7659999999999998e-06 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.011271+0000", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6049999999999997e-06 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023721+0000", 2024-09-19T21:37:45.740 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.012450016 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023727+0000", 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025151+0000", 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014244670000001 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.741 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025166+0000", 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4983e-05 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4174 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 74.626109974000002, 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046904990000001, 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.742 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4174 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.743 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020519+0000", 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020517+0000", 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999981 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020528+0000", 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0638999999999999e-05 2024-09-19T21:37:45.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020557+0000", 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9649999999999999e-05 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020570+0000", 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.312e-05 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.745 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023746+0000", 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031754330000000001 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023751+0000", 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2560000000000002e-06 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025195+0000", 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0014441590000001 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.746 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025209+0000", 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4171999999999999e-05 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.5967.0:4175 3.1d 3.517d1c3d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 74.626037549000003, 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0046400499999999, 2024-09-19T21:37:45.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.5967", 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.122:0/2798939085", 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 4175 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.748 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020591+0000", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020596+0000", 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.711e-06 2024-09-19T21:37:45.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020605+0000", 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6379999999999993e-06 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.020612+0000", 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5140000000000004e-06 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.750 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023770+0000", 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0031572509999999998 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for peered", 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:31.023775+0000", 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9520000000000002e-06 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025226+0000", 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.001451793 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-09-19T21:37:45.751 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-09-19T21:36:32.025231+0000", 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8679999999999998e-06 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-09-19T21:37:45.752 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:37:45.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:45.754 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:45.749+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:45.855 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:45.849+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:45.871 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:45.871 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:45.871 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:45.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:45.955 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:45.949+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:45.985 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:45.985 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:45.985 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:45.985 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:45.985 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:45.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:46.056 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:46.054+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:46.150 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8616 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13383680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895874443, 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055603580999999999, 2024-09-19T21:37:46.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8616 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253348+0000", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253347+0000", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253349+0000", 2024-09-19T21:37:46.156 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.1909999999999999e-06 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253354+0000", 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5859999999999998e-06 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253358+0000", 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6959999999999998e-06 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.157 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308816+0000", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055458615000000003 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308861+0000", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4645999999999999e-05 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308951+0000", 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0185999999999993e-05 2024-09-19T21:37:46.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8617 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13398016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89586090399999, 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055787128999999998, 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.159 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8617 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:46.160 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253361+0000", 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253363+0000", 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.2879999999999999e-06 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.161 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253376+0000", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3298999999999999e-05 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253380+0000", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0749999999999999e-06 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.308992+0000", 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055611134 2024-09-19T21:37:46.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309062+0000", 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0522e-05 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309148+0000", 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.6155000000000004e-05 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8618 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13412352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895835164, 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055914770000000003, 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8618 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253387+0000", 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.165 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253386+0000", 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253389+0000", 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7269999999999999e-06 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253394+0000", 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2569999999999998e-06 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.166 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253398+0000", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.6440000000000002e-06 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309187+0000", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055788817999999997 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309246+0000", 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9604000000000001e-05 2024-09-19T21:37:46.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309302+0000", 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5118000000000003e-05 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8619 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13426688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89582095, 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056090482999999997, 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8619 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.169 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253401+0000", 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253403+0000", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.357e-06 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253408+0000", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.5220000000000001e-06 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253411+0000", 2024-09-19T21:37:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8809999999999998e-06 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309336+0000", 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055924245999999997 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309401+0000", 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5667999999999995e-05 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.172 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309492+0000", 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.0147000000000003e-05 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8620 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13441024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895800033, 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056216808, 2024-09-19T21:37:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8620 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253422+0000", 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.175 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253424+0000", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3580000000000001e-06 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253430+0000", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5439999999999998e-06 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253434+0000", 2024-09-19T21:37:46.176 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0910000000000003e-06 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309514+0000", 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056080491000000003 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309568+0000", 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4143e-05 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.177 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309639+0000", 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0513999999999996e-05 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8621 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13455360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89578465700001, 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056385228000000003, 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8621 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.179 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253437+0000", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253439+0000", 2024-09-19T21:37:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.221e-06 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253443+0000", 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2930000000000002e-06 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253447+0000", 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9519999999999996e-06 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.181 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309680+0000", 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056232624000000002 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309746+0000", 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.5933999999999996e-05 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309823+0000", 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6582999999999995e-05 2024-09-19T21:37:46.182 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8622 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13469696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89576972899999, 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056529306000000001, 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8622 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253452+0000", 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253455+0000", 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.433e-06 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.185 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253464+0000", 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.3110000000000006e-06 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253482+0000", 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8403999999999999e-05 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309840+0000", 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056357668 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.186 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309912+0000", 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2455999999999999e-05 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.309982+0000", 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9399000000000004e-05 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8623 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13484032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895717109, 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056685773000000002, 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8623 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253505+0000", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253504+0000", 2024-09-19T21:37:46.189 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253508+0000", 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0350000000000003e-06 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253515+0000", 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0269999999999998e-06 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.190 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253541+0000", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6033999999999998e-05 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310019+0000", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056478222000000002 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310091+0000", 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1942999999999999e-05 2024-09-19T21:37:46.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310191+0000", 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9302999999999994e-05 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8624 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13498368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89566753099999, 2024-09-19T21:37:46.192 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056782510000000001, 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8624 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253554+0000", 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253549+0000", 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999943 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.194 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253558+0000", 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.901e-06 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253564+0000", 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3860000000000004e-06 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253568+0000", 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1699999999999999e-06 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310228+0000", 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056660275000000003 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310284+0000", 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5140999999999999e-05 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310337+0000", 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3396999999999997e-05 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8625 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13512704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895650316, 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056952410000000002, 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8625 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253572+0000", 2024-09-19T21:37:46.198 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253571+0000", 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253574+0000", 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.3269999999999999e-06 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.199 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253578+0000", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.605e-06 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253582+0000", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7629999999999998e-06 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310371+0000", 2024-09-19T21:37:46.200 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056788582999999997 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310439+0000", 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8555999999999999e-05 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310524+0000", 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4888000000000004e-05 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8626 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13527040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.895631178, 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057115549000000002, 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8626 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.202 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253591+0000", 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253590+0000", 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253593+0000", 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.6400000000000001e-06 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253598+0000", 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1680000000000003e-06 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253602+0000", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9570000000000004e-06 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310550+0000", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056947679000000001 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310621+0000", 2024-09-19T21:37:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1438000000000003e-05 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310706+0000", 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5017000000000003e-05 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8627 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13541376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89561633, 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057277634000000001, 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8627 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253606+0000", 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253605+0000", 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253615+0000", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4779999999999997e-06 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253621+0000", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0569999999999998e-06 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253625+0000", 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.1529999999999999e-06 2024-09-19T21:37:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310745+0000", 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057119830000000003 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310806+0000", 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0859000000000001e-05 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310883+0000", 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7597999999999998e-05 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8628 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13555712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89559333700001, 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057410052000000003, 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8628 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253629+0000", 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253628+0000", 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253631+0000", 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.4739999999999999e-06 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253635+0000", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6410000000000001e-06 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253639+0000", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.76e-06 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310916+0000", 2024-09-19T21:37:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.057276588000000003 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.310978+0000", 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2176999999999998e-05 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311039+0000", 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0806e-05 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8629 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13570048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.894861045, 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056862338999999998, 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8629 2024-09-19T21:37:46.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254361+0000", 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.253642+0000", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9992809 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254367+0000", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00072535999999999998 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254388+0000", 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.0854000000000001e-05 2024-09-19T21:37:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254399+0000", 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.117e-05 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311079+0000", 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056679633 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.219 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311150+0000", 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0605999999999995e-05 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311223+0000", 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3798000000000001e-05 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8630 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13584384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.220 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89481073499999, 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056944251000000001, 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8630 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.221 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254411+0000", 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254415+0000", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.7689999999999998e-06 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254426+0000", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.136e-05 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254430+0000", 2024-09-19T21:37:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4410000000000003e-06 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311249+0000", 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056818408000000001 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311302+0000", 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.312e-05 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311356+0000", 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.3671000000000002e-05 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8631 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13598720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89474842300001, 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.05708448, 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8631 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254474+0000", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254435+0000", 2024-09-19T21:37:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999609 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254479+0000", 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4567000000000002e-05 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254496+0000", 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7178999999999999e-05 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254506+0000", 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5840000000000001e-06 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311393+0000", 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056887031999999997 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311473+0000", 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0249999999999999e-05 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311558+0000", 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4912000000000003e-05 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8632 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13613056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89341779, 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055933666, 2024-09-19T21:37:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8632 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255804+0000", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.254514+0000", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9987097 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255812+0000", 2024-09-19T21:37:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.001298022 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255856+0000", 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.4499999999999997e-05 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255927+0000", 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.1044000000000006e-05 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311584+0000", 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055656938000000003 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311670+0000", 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.5919000000000002e-05 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311738+0000", 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.7615999999999998e-05 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8633 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 13627392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.893282212, 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055989013999999997, 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8633 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255940+0000", 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255939+0000", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255945+0000", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.3759999999999996e-06 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255956+0000", 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.0607e-05 2024-09-19T21:37:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.255970+0000", 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4584e-05 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311777+0000", 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055806260000000003 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311849+0000", 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2092e-05 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311929+0000", 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.0031999999999995e-05 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8816 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12582912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89320545300001, 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056051851, 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8816 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256017+0000", 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256016+0000", 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256030+0000", 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4628e-05 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256055+0000", 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5171e-05 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256067+0000", 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1141000000000001e-05 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.311957+0000", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055890084999999999 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312008+0000", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.1146000000000001e-05 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312068+0000", 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0593000000000001e-05 2024-09-19T21:37:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5043.0:8817 3.2 3:43167c50:::rbd_data.11066d5522c8.0000000000000003:12 [read 12597248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 151.89314672899999, 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.056180873999999999, 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5043", 2024-09-19T21:37:46.244 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/1245438650", 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 8817 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256075+0000", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256079+0000", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.758e-06 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256105+0000", 2024-09-19T21:37:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5480999999999999e-05 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.256123+0000", 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8227e-05 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312108+0000", 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.055984691000000003 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312180+0000", 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2044000000000003e-05 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:14.312256+0000", 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6437e-05 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.248 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:46.250 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:46.246+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:46.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:46.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:46.345+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:46.372 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "num_ops": 0 2024-09-19T21:37:46.372 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:46.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:46.451 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:46.450+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:46.538 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:46.538 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [], 2024-09-19T21:37:46.538 INFO:teuthology.orchestra.run.smithi033.stdout: "complaint_time": 30, 2024-09-19T21:37:46.538 INFO:teuthology.orchestra.run.smithi033.stdout: "num_blocked_ops": 0 2024-09-19T21:37:46.538 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:46.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:46.551 INFO:tasks.ceph.osd.4.smithi033.stderr:2024-09-19T21:37:46.545+0000 7f843e24f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 21025) UID: 0 2024-09-19T21:37:46.617 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T21:37:46.652 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:46.650+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "size": 20, 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 600, 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "ops": [ 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4116 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.46062938, 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070548815000000001, 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4116 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.664 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197693+0000", 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197696+0000", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.1939999999999998e-06 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197705+0000", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.9490000000000002e-06 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.197710+0000", 2024-09-19T21:37:46.666 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.4310000000000002e-06 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267685+0000", 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.06997507 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.267829+0000", 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014381499999999999 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.667 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268242+0000", 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00041281800000000002 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4118 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.46004946900001, 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070544071999999999, 2024-09-19T21:37:46.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4118 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:46.669 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198273+0000", 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198272+0000", 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198279+0000", 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9190000000000002e-06 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198295+0000", 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6044000000000001e-05 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.671 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198303+0000", 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.4619999999999996e-06 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268319+0000", 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070015849000000005 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.672 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268427+0000", 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000107629 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268817+0000", 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00039029199999999998 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4125 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45998682199999, 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070967631000000003, 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.674 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4125 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.675 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198336+0000", 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198335+0000", 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198338+0000", 2024-09-19T21:37:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0520000000000002e-06 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198384+0000", 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.6007999999999997e-05 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198392+0000", 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4959999999999999e-06 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268867+0000", 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070475324000000006 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.268937+0000", 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9362000000000004e-05 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269303+0000", 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036680099999999999 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.679 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4127 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1114112~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45992050199999, 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071403578999999995, 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4127 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198402+0000", 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198401+0000", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198406+0000", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.2400000000000001e-06 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198436+0000", 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9859999999999999e-05 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.682 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.198447+0000", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1541e-05 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269332+0000", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070885260000000005 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269399+0000", 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.6940000000000006e-05 2024-09-19T21:37:46.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269806+0000", 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040631199999999998 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4138 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 2064384~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45921682400001, 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071393360000000003, 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4138 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.685 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199106+0000", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199105+0000", 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199108+0000", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.9019999999999999e-06 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199136+0000", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.7458e-05 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199141+0000", 2024-09-19T21:37:46.687 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.0069999999999997e-06 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.269869+0000", 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070728183 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270006+0000", 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013661599999999999 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.688 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270499+0000", 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00049350900000000005 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4147 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45914400199999, 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071961368999999997, 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4147 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199179+0000", 2024-09-19T21:37:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199178+0000", 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199181+0000", 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.48e-06 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199185+0000", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.3120000000000004e-06 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199188+0000", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.0620000000000001e-06 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270548+0000", 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071360260999999994 2024-09-19T21:37:46.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.270693+0000", 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014426099999999999 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271140+0000", 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00044725100000000002 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4152 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45913026400001, 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072408042000000006, 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4152 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:46.695 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199192+0000", 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199195+0000", 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.5150000000000001e-06 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.696 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199201+0000", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2949999999999999e-06 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199207+0000", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.772e-06 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271179+0000", 2024-09-19T21:37:46.697 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.071972724000000002 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271244+0000", 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.4603999999999993e-05 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271600+0000", 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000356366 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4155 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45911257500001, 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072915209999999994, 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.699 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4155 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199210+0000", 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199212+0000", 2024-09-19T21:37:46.701 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.001e-06 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199215+0000", 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.2949999999999998e-06 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199219+0000", 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4.0130000000000004e-06 2024-09-19T21:37:46.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271631+0000", 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072411726999999995 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.271758+0000", 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012683900000000001 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272125+0000", 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00036758199999999998 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4159 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45906218900001, 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073539924000000007, 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.704 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4159 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199260+0000", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199263+0000", 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.3459999999999998e-06 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199269+0000", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.0399999999999998e-06 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199273+0000", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8179999999999997e-06 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272161+0000", 2024-09-19T21:37:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.072888254999999999 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272239+0000", 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.7812000000000006e-05 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272800+0000", 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00056115899999999999 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4163 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45903530000001, 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074022092999999997, 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4163 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199287+0000", 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967296 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199289+0000", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.8169999999999999e-06 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199292+0000", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4209999999999999e-06 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199295+0000", 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8729999999999999e-06 2024-09-19T21:37:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272838+0000", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073542709999999997 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.272900+0000", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.2052999999999997e-05 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273309+0000", 2024-09-19T21:37:46.712 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00040944700000000001 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4164 3.1 3:81e59c28:::rbd_data.1106df37d7f5.0000000000000003:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45899472400001, 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074620120999999998, 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4164 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199328+0000", 2024-09-19T21:37:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199327+0000", 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199331+0000", 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.8210000000000003e-06 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199338+0000", 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.9909999999999996e-06 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199344+0000", 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.8019999999999997e-06 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273337+0000", 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.073993054000000003 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.716 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273438+0000", 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000101584 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273948+0000", 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00050951600000000001 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4166 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45897310999999, 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075481258999999995, 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4166 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.718 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199349+0000", 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199352+0000", 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.4750000000000002e-06 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199358+0000", 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1879999999999999e-06 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199364+0000", 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9440000000000002e-06 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.273995+0000", 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.074630646999999994 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.721 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274067+0000", 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.2385999999999994e-05 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274831+0000", 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00076336299999999998 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5154.0:4167 3.1 3:838d7f4f:::rbd_data.1106df37d7f5.0000000000000005:14 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-19T21:37:46.722 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 138.45895391100001, 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075882569999999996, 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5154", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2528023984", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 4167 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199369+0000", 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199368+0000", 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199371+0000", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.8629999999999999e-06 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199377+0000", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.5559999999999998e-06 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.199382+0000", 2024-09-19T21:37:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.322e-06 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274867+0000", 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.075484669000000004 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.274937+0000", 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.0714000000000001e-05 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:28.275251+0000", 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031382200000000001 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22508 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 606208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 124.96963630899999, 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070434904000000007, 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22508 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:46.729 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688686+0000", 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688685+0000", 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.730 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688689+0000", 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9249999999999997e-06 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688697+0000", 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.3470000000000001e-06 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.688706+0000", 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.6150000000000003e-06 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758805+0000", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070098270000000004 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.758887+0000", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.2482000000000001e-05 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759121+0000", 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00023398000000000001 2024-09-19T21:37:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22511 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 632832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 124.969012827, 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070319839999999995, 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22511 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.734 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689310+0000", 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689309+0000", 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999995 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689313+0000", 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 3.9999999999999998e-06 2024-09-19T21:37:46.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689320+0000", 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.4569999999999999e-06 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689326+0000", 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.2800000000000003e-06 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759440+0000", 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070114470999999998 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759527+0000", 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.7176999999999996e-05 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759630+0000", 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00010217000000000001 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.737 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.5316.0:22512 3.19 3:9ef8d79f:::rbd_data.1106b7b0d98d.0000000000000000:18 [read 646144~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e67)", 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 124.96868055100001, 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070237003000000006, 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.5316", 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/2804959817", 2024-09-19T21:37:46.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 22512 2024-09-19T21:37:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689642+0000", 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.741 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689330+0000", 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9996881 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689647+0000", 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000316996 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.742 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689656+0000", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4080000000000004e-06 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.689663+0000", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.8650000000000003e-06 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759686+0000", 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.070023022000000004 2024-09-19T21:37:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759775+0000", 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 8.8201000000000005e-05 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:35:41.759879+0000", 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000104443 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.744 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:72 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [read 7258112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.879589334000002, 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28279926599999999, 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 72 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.745 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778733+0000", 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.746 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778732+0000", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.9999986 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778741+0000", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.4669999999999993e-06 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778759+0000", 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7927999999999998e-05 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778772+0000", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2666e-05 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778848+0000", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 7.6297000000000005e-05 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.778858+0000", 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.8700000000000004e-06 2024-09-19T21:37:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.060879+0000", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.28202089600000002 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061222+0000", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00034282300000000002 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061532+0000", 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00031068699999999997 2024-09-19T21:37:46.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:83 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 0~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.856992978999997, 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.260687108, 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:46.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 83 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801330+0000", 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801329+0000", 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801335+0000", 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 5.9750000000000004e-06 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.752 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801349+0000", 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.3896e-05 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801360+0000", 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.19e-05 2024-09-19T21:37:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801501+0000", 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014054900000000001 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801518+0000", 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.7091e-05 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.754 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061591+0000", 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26007303999999998 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.061737+0000", 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00014578099999999999 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062017+0000", 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00027974500000000003 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:90 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.856662311999997, 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26073563900000002, 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 90 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801660+0000", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801659+0000", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801665+0000", 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.066e-06 2024-09-19T21:37:46.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801682+0000", 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.6867999999999998e-05 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801695+0000", 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.2751e-05 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.759 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801790+0000", 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.5434999999999997e-05 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.801806+0000", 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.5877000000000001e-05 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062136+0000", 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26032992700000002 2024-09-19T21:37:46.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062296+0000", 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00015967800000000001 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062396+0000", 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 9.9894000000000003e-05 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "description": "osd_op(client.6063.0:103 3.8 3:14903ab4:::rbd_data.1106f44e6b3b.0000000000000005:head [sparse-read 131072~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "initiated_at": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 69.856142739999996, 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26072235900000001, 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type_data": { 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flag_point": "started", 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "client_info": { 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "client": "client.6063", 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "client_addr": "172.21.15.122:0/344081198", 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tid": 103 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: "events": [ 2024-09-19T21:37:46.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "initiated", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "throttled", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802180+0000", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "header_read", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802179+0000", 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 4294967295.999999 2024-09-19T21:37:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "all_read", 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802185+0000", 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 6.1319999999999999e-06 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "dispatched", 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802200+0000", 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.4585e-05 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "queued_for_pg", 2024-09-19T21:37:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802211+0000", 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 1.1368e-05 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802351+0000", 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00013956799999999999 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "waiting for peered", 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:36.802371+0000", 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 2.084e-05 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.765 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "reached_pg", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062590+0000", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.26021870699999999 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "started", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062710+0000", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.00012004499999999999 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "event": "done", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "time": "2024-09-19T21:36:37.062902+0000", 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: "duration": 0.000191977 2024-09-19T21:37:46.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-19T21:37:46.767 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-19T21:37:46.767 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-19T21:37:46.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:46.769 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:46.766+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:46.769 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-09-19T21:37:46.791 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-09-19T21:37:46.813 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-09-19T21:37:46.840 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-09-19T21:37:46.845 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 2, 5] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-19T21:37:46.845 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T21:37:46.845 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-19T21:37:46.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-19T21:37:46.862 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-09-19T21:37:46.869 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:46.866+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:46.883 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-09-19T21:37:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T21:37:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T21:37:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T21:37:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T21:37:46.885 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T21:37:46.886 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T21:37:46.887 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:46.887 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:46.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:46.969 INFO:tasks.ceph.osd.2.smithi012.stderr:2024-09-19T21:37:46.966+0000 7fe5e23dc700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 20945) UID: 0 2024-09-19T21:37:47.009 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.009 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:47.070 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:47.065+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:47.131 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.131 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T21:37:47.171 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-09-19T21:37:47.170+0000 7eff847dd700 -1 mon.a@0(leader).osd e172 definitely_dead 0 2024-09-19T21:37:47.171 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-19T21:37:47.170+0000 7fb874bd5700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 20942) UID: 0 2024-09-19T21:37:47.246 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.247 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T21:37:47.272 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-09-19T21:37:47.265+0000 7f958b017700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 21021) UID: 0 2024-09-19T21:37:47.346 INFO:teuthology.orchestra.run.smithi122.stdout:All operations completed A-OK! 2024-09-19T21:37:47.374 INFO:tasks.ceph.osd.5.smithi033.stderr:2024-09-19T21:37:47.369+0000 7fb8a8da3700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 21023) UID: 0 2024-09-19T21:37:47.382 DEBUG:teuthology.parallel:result is None 2024-09-19T21:37:47.383 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-19T21:37:47.393 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-19T21:37:47.426 INFO:tasks.thrashosds:joining thrashosds 2024-09-19T21:37:47.427 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.427 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T21:37:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.537 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T21:37:47.653 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.653 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T21:37:47.775 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:47.776 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:47.786 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T21:37:47.862 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.4. 2024-09-19T21:37:48.083 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-19T21:37:48.083 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.083 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.084 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T21:37:48.200 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.200 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T21:37:48.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T21:37:48.319 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.319 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T21:37:48.436 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.436 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T21:37:48.563 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.563 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T21:37:48.683 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.683 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T21:37:48.801 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.801 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T21:37:48.862 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-09-19T21:37:48.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T21:37:48.928 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:48.928 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:48.938 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T21:37:49.042 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:49.042 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:49.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T21:37:49.162 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:49.162 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:49.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T21:37:49.278 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:49.278 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:49.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T21:37:49.397 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:49.397 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:49.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T21:37:49.515 INFO:teuthology.orchestra.run.smithi033.stdout:ERROR: (22) Invalid argument 2024-09-19T21:37:49.515 INFO:teuthology.orchestra.run.smithi033.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T21:37:49.872 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-09-19T21:37:53.083 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-19T21:37:53.229 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a: {} 2024-09-19T21:37:53.244 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b: {} 2024-09-19T21:37:53.257 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c: {} 2024-09-19T21:37:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:37:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:37:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T21:37:53.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-19T21:37:53.868 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-09-19T21:37:53.883 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-19T21:37:54.896 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-19T21:37:54.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-19T21:37:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-09-19T21:37:55.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-19T21:37:56.906 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-19T21:37:56.917 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-19T21:37:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.2 to 1 (10000) 2024-09-19T21:37:57.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-19T21:37:58.931 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-19T21:37:58.943 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-19T21:37:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-09-19T21:37:59.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-19T21:38:00.936 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-19T21:38:00.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-19T21:38:01.952 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-09-19T21:38:01.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-19T21:38:02.968 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-19T21:38:02.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-19T21:38:03.980 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.5 to 1 (10000) 2024-09-19T21:38:03.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-19T21:38:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-19T21:38:05.007 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-19T21:38:05.007 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-19T21:38:05.007 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-19T21:38:05.007 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-19T21:38:05.007 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T21:38:05.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:38:05.322 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:05.323 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":190,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:38:04.976085+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:37:47.840357+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T21:34:19.884271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-19T21:34:28.955849+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.4","osds":[5,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:38:05.338 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T21:38:05.338 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:38:05.639 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:05.639 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":190,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:38:04.976085+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:37:47.840357+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T21:34:19.884271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-19T21:34:28.955849+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.4","osds":[5,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:38:05.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T21:38:05.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T21:38:05.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T21:38:05.660 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T21:38:05.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T21:38:05.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T21:38:05.849 INFO:teuthology.orchestra.run.smithi012.stdout:51539607602 2024-09-19T21:38:05.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T21:38:05.852 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:05.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T21:38:05.854 INFO:teuthology.orchestra.run.smithi012.stdout:55834574897 2024-09-19T21:38:05.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T21:38:05.862 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:05.862 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T21:38:05.868 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:05.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T21:38:05.880 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:05.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T21:38:06.193 INFO:teuthology.orchestra.run.smithi012.stdout:55834574898 2024-09-19T21:38:06.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.0 2024-09-19T21:38:06.265 INFO:teuthology.orchestra.run.smithi012.stdout:55834574898 2024-09-19T21:38:06.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.1 2024-09-19T21:38:06.307 INFO:teuthology.orchestra.run.smithi012.stdout:55834574896 2024-09-19T21:38:06.329 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574896 for osd.2 2024-09-19T21:38:06.333 INFO:teuthology.orchestra.run.smithi012.stdout:51539607601 2024-09-19T21:38:06.345 INFO:teuthology.orchestra.run.smithi012.stdout:55834574898 2024-09-19T21:38:06.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607602 got 51539607601 for osd.3 2024-09-19T21:38:06.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.5 2024-09-19T21:38:06.378 INFO:teuthology.orchestra.run.smithi012.stdout:55834574898 2024-09-19T21:38:06.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574898 for osd.4 2024-09-19T21:38:07.212 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T21:38:07.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T21:38:07.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T21:38:07.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T21:38:07.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T21:38:07.393 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T21:38:07.639 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:07.647 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:07.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.0 2024-09-19T21:38:07.661 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.1 2024-09-19T21:38:07.662 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.711 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:07.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.5 2024-09-19T21:38:07.724 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.730 INFO:teuthology.orchestra.run.smithi012.stdout:55834574897 2024-09-19T21:38:07.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574897 got 55834574897 for osd.2 2024-09-19T21:38:07.744 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.764 INFO:teuthology.orchestra.run.smithi012.stdout:51539607603 2024-09-19T21:38:07.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607602 got 51539607603 for osd.3 2024-09-19T21:38:07.778 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.808 INFO:teuthology.orchestra.run.smithi012.stdout:55834574899 2024-09-19T21:38:07.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574899 got 55834574899 for osd.4 2024-09-19T21:38:07.830 DEBUG:teuthology.parallel:result is None 2024-09-19T21:38:07.830 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-19T21:38:07.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:08.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:08.164 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-19T21:38:06.863042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307848,"kb_used_data":14496,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923192,"statfs":{"total":579820584960,"available":579505348608,"internally_reserved":0,"allocated":14843904,"data_stored":6869286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001290"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4855,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794547+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.794547+0000","last_peered":"2024-09-19T21:38:04.794547+0000","last_clean":"2024-09-19T21:38:04.794547+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.794547+0000","last_undegraded":"2024-09-19T21:38:04.794547+0000","last_fullsized":"2024-09-19T21:38:04.794547+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10522,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972258+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:03.972258+0000","last_peered":"2024-09-19T21:38:03.972258+0000","last_clean":"2024-09-19T21:38:03.972258+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:03.972258+0000","last_undegraded":"2024-09-19T21:38:03.972258+0000","last_fullsized":"2024-09-19T21:38:03.972258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16708,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796158+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:04.796158+0000","last_peered":"2024-09-19T21:38:04.796158+0000","last_clean":"2024-09-19T21:38:04.796158+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:04.796158+0000","last_undegraded":"2024-09-19T21:38:04.796158+0000","last_fullsized":"2024-09-19T21:38:04.796158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9301,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795882+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:04.795882+0000","last_peered":"2024-09-19T21:38:04.795882+0000","last_clean":"2024-09-19T21:38:04.795882+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:04.795882+0000","last_undegraded":"2024-09-19T21:38:04.795882+0000","last_fullsized":"2024-09-19T21:38:04.795882+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14870,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793429+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:04.793429+0000","last_peered":"2024-09-19T21:38:04.793429+0000","last_clean":"2024-09-19T21:38:04.793429+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:04.793429+0000","last_undegraded":"2024-09-19T21:38:04.793429+0000","last_fullsized":"2024-09-19T21:38:04.793429+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6442,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794464+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.794464+0000","last_peered":"2024-09-19T21:38:04.794464+0000","last_clean":"2024-09-19T21:38:04.794464+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.794464+0000","last_undegraded":"2024-09-19T21:38:04.794464+0000","last_fullsized":"2024-09-19T21:38:04.794464+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972414+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:03.972414+0000","last_peered":"2024-09-19T21:38:03.972414+0000","last_clean":"2024-09-19T21:38:03.972414+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:03.972414+0000","last_undegraded":"2024-09-19T21:38:03.972414+0000","last_fullsized":"2024-09-19T21:38:03.972414+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7034,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.974149+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:03.974149+0000","last_peered":"2024-09-19T21:38:03.974149+0000","last_clean":"2024-09-19T21:38:03.974149+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:03.974149+0000","last_undegraded":"2024-09-19T21:38:03.974149+0000","last_fullsized":"2024-09-19T21:38:03.974149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10396,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795908+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:04.795908+0000","last_peered":"2024-09-19T21:38:04.795908+0000","last_clean":"2024-09-19T21:38:04.795908+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:04.795908+0000","last_undegraded":"2024-09-19T21:38:04.795908+0000","last_fullsized":"2024-09-19T21:38:04.795908+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796176+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:04.796176+0000","last_peered":"2024-09-19T21:38:04.796176+0000","last_clean":"2024-09-19T21:38:04.796176+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:04.796176+0000","last_undegraded":"2024-09-19T21:38:04.796176+0000","last_fullsized":"2024-09-19T21:38:04.796176+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9034,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793259+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:04.793259+0000","last_peered":"2024-09-19T21:38:04.793259+0000","last_clean":"2024-09-19T21:38:04.793259+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:04.793259+0000","last_undegraded":"2024-09-19T21:38:04.793259+0000","last_fullsized":"2024-09-19T21:38:04.793259+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793436+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:04.793436+0000","last_peered":"2024-09-19T21:38:04.793436+0000","last_clean":"2024-09-19T21:38:04.793436+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:04.793436+0000","last_undegraded":"2024-09-19T21:38:04.793436+0000","last_fullsized":"2024-09-19T21:38:04.793436+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9090,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793496+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:04.793496+0000","last_peered":"2024-09-19T21:38:04.793496+0000","last_clean":"2024-09-19T21:38:04.793496+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:04.793496+0000","last_undegraded":"2024-09-19T21:38:04.793496+0000","last_fullsized":"2024-09-19T21:38:04.793496+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796031+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:04.796031+0000","last_peered":"2024-09-19T21:38:04.796031+0000","last_clean":"2024-09-19T21:38:04.796031+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:04.796031+0000","last_undegraded":"2024-09-19T21:38:04.796031+0000","last_fullsized":"2024-09-19T21:38:04.796031+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13379,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794209+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.794209+0000","last_peered":"2024-09-19T21:38:04.794209+0000","last_clean":"2024-09-19T21:38:04.794209+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.794209+0000","last_undegraded":"2024-09-19T21:38:04.794209+0000","last_fullsized":"2024-09-19T21:38:04.794209+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":190,"state":"peering","last_fresh":"2024-09-19T21:38:04.990617+0000","last_change":"2024-09-19T21:37:05.846511+0000","last_active":"2024-09-19T21:37:05.846255+0000","last_peered":"2024-09-19T21:37:04.841482+0000","last_clean":"2024-09-19T21:37:04.841482+0000","last_became_active":"2024-09-19T21:36:32.030278+0000","last_became_peered":"2024-09-19T21:36:32.030278+0000","last_unstale":"2024-09-19T21:38:04.990617+0000","last_undegraded":"2024-09-19T21:38:04.990617+0000","last_fullsized":"2024-09-19T21:38:04.990617+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5695,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972153+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:03.972153+0000","last_peered":"2024-09-19T21:38:03.972153+0000","last_clean":"2024-09-19T21:38:03.972153+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:03.972153+0000","last_undegraded":"2024-09-19T21:38:03.972153+0000","last_fullsized":"2024-09-19T21:38:03.972153+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10099,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972983+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:03.972983+0000","last_peered":"2024-09-19T21:38:03.972983+0000","last_clean":"2024-09-19T21:38:03.972983+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:03.972983+0000","last_undegraded":"2024-09-19T21:38:03.972983+0000","last_fullsized":"2024-09-19T21:38:03.972983+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":234,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972272+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:03.972272+0000","last_peered":"2024-09-19T21:38:03.972272+0000","last_clean":"2024-09-19T21:38:03.972272+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:03.972272+0000","last_undegraded":"2024-09-19T21:38:03.972272+0000","last_fullsized":"2024-09-19T21:38:03.972272+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12092,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794181+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.794181+0000","last_peered":"2024-09-19T21:38:04.794181+0000","last_clean":"2024-09-19T21:38:04.794181+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.794181+0000","last_undegraded":"2024-09-19T21:38:04.794181+0000","last_fullsized":"2024-09-19T21:38:04.794181+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9237,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793212+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:04.793212+0000","last_peered":"2024-09-19T21:38:04.793212+0000","last_clean":"2024-09-19T21:38:04.793212+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:04.793212+0000","last_undegraded":"2024-09-19T21:38:04.793212+0000","last_fullsized":"2024-09-19T21:38:04.793212+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.973469+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:03.973469+0000","last_peered":"2024-09-19T21:38:03.973469+0000","last_clean":"2024-09-19T21:38:03.973469+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:03.973469+0000","last_undegraded":"2024-09-19T21:38:03.973469+0000","last_fullsized":"2024-09-19T21:38:03.973469+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":211,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796057+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:04.796057+0000","last_peered":"2024-09-19T21:38:04.796057+0000","last_clean":"2024-09-19T21:38:04.796057+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:04.796057+0000","last_undegraded":"2024-09-19T21:38:04.796057+0000","last_fullsized":"2024-09-19T21:38:04.796057+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":418,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795727+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:04.795727+0000","last_peered":"2024-09-19T21:38:04.795727+0000","last_clean":"2024-09-19T21:38:04.795727+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:04.795727+0000","last_undegraded":"2024-09-19T21:38:04.795727+0000","last_fullsized":"2024-09-19T21:38:04.795727+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15962,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793157+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:04.793157+0000","last_peered":"2024-09-19T21:38:04.793157+0000","last_clean":"2024-09-19T21:38:04.793157+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:04.793157+0000","last_undegraded":"2024-09-19T21:38:04.793157+0000","last_fullsized":"2024-09-19T21:38:04.793157+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5835,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972035+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:03.972035+0000","last_peered":"2024-09-19T21:38:03.972035+0000","last_clean":"2024-09-19T21:38:03.972035+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:03.972035+0000","last_undegraded":"2024-09-19T21:38:03.972035+0000","last_fullsized":"2024-09-19T21:38:03.972035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9163,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972162+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:03.972162+0000","last_peered":"2024-09-19T21:38:03.972162+0000","last_clean":"2024-09-19T21:38:03.972162+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:03.972162+0000","last_undegraded":"2024-09-19T21:38:03.972162+0000","last_fullsized":"2024-09-19T21:38:03.972162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4050,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.973382+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:03.973382+0000","last_peered":"2024-09-19T21:38:03.973382+0000","last_clean":"2024-09-19T21:38:03.973382+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:03.973382+0000","last_undegraded":"2024-09-19T21:38:03.973382+0000","last_fullsized":"2024-09-19T21:38:03.973382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9705,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795984+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:04.795984+0000","last_peered":"2024-09-19T21:38:04.795984+0000","last_clean":"2024-09-19T21:38:04.795984+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:04.795984+0000","last_undegraded":"2024-09-19T21:38:04.795984+0000","last_fullsized":"2024-09-19T21:38:04.795984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6431,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972573+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:03.972573+0000","last_peered":"2024-09-19T21:38:03.972573+0000","last_clean":"2024-09-19T21:38:03.972573+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:03.972573+0000","last_undegraded":"2024-09-19T21:38:03.972573+0000","last_fullsized":"2024-09-19T21:38:03.972573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5591,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972984+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:03.972984+0000","last_peered":"2024-09-19T21:38:03.972984+0000","last_clean":"2024-09-19T21:38:03.972984+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:03.972984+0000","last_undegraded":"2024-09-19T21:38:03.972984+0000","last_fullsized":"2024-09-19T21:38:03.972984+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8791,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795727+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:04.795727+0000","last_peered":"2024-09-19T21:38:04.795727+0000","last_clean":"2024-09-19T21:38:04.795727+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:04.795727+0000","last_undegraded":"2024-09-19T21:38:04.795727+0000","last_fullsized":"2024-09-19T21:38:04.795727+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.971758+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:03.971758+0000","last_peered":"2024-09-19T21:38:03.971758+0000","last_clean":"2024-09-19T21:38:03.971758+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:03.971758+0000","last_undegraded":"2024-09-19T21:38:03.971758+0000","last_fullsized":"2024-09-19T21:38:03.971758+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7485,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794289+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.794289+0000","last_peered":"2024-09-19T21:38:04.794289+0000","last_clean":"2024-09-19T21:38:04.794289+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.794289+0000","last_undegraded":"2024-09-19T21:38:04.794289+0000","last_fullsized":"2024-09-19T21:38:04.794289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7910,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795846+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:04.795846+0000","last_peered":"2024-09-19T21:38:04.795846+0000","last_clean":"2024-09-19T21:38:04.795846+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:04.795846+0000","last_undegraded":"2024-09-19T21:38:04.795846+0000","last_fullsized":"2024-09-19T21:38:04.795846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33928,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337912,"statfs":{"total":96636764160,"available":96602021888,"internally_reserved":0,"allocated":2871296,"data_stored":1538466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.997}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.93700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55472,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316368,"statfs":{"total":96636764160,"available":96579960832,"internally_reserved":0,"allocated":2265088,"data_stored":943128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.96399999999999997}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.179}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.109},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2273280,"data_stored":948079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.89800000000000002}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.747}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.873}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2871296,"data_stored":1538466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2277376,"data_stored":948098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.82999999999999996}]}]},{"osd":3,"up_from":12,"seq":51539607603,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.3009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:08.168 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:08.475 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:08.479 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:08.492 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-19T21:38:06.863042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307848,"kb_used_data":14496,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923192,"statfs":{"total":579820584960,"available":579505348608,"internally_reserved":0,"allocated":14843904,"data_stored":6869286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001290"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4855,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794547+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.794547+0000","last_peered":"2024-09-19T21:38:04.794547+0000","last_clean":"2024-09-19T21:38:04.794547+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.794547+0000","last_undegraded":"2024-09-19T21:38:04.794547+0000","last_fullsized":"2024-09-19T21:38:04.794547+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10522,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972258+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:03.972258+0000","last_peered":"2024-09-19T21:38:03.972258+0000","last_clean":"2024-09-19T21:38:03.972258+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:03.972258+0000","last_undegraded":"2024-09-19T21:38:03.972258+0000","last_fullsized":"2024-09-19T21:38:03.972258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16708,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796158+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:04.796158+0000","last_peered":"2024-09-19T21:38:04.796158+0000","last_clean":"2024-09-19T21:38:04.796158+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:04.796158+0000","last_undegraded":"2024-09-19T21:38:04.796158+0000","last_fullsized":"2024-09-19T21:38:04.796158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9301,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795882+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:04.795882+0000","last_peered":"2024-09-19T21:38:04.795882+0000","last_clean":"2024-09-19T21:38:04.795882+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:04.795882+0000","last_undegraded":"2024-09-19T21:38:04.795882+0000","last_fullsized":"2024-09-19T21:38:04.795882+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14870,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793429+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:04.793429+0000","last_peered":"2024-09-19T21:38:04.793429+0000","last_clean":"2024-09-19T21:38:04.793429+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:04.793429+0000","last_undegraded":"2024-09-19T21:38:04.793429+0000","last_fullsized":"2024-09-19T21:38:04.793429+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6442,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794464+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.794464+0000","last_peered":"2024-09-19T21:38:04.794464+0000","last_clean":"2024-09-19T21:38:04.794464+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.794464+0000","last_undegraded":"2024-09-19T21:38:04.794464+0000","last_fullsized":"2024-09-19T21:38:04.794464+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972414+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:03.972414+0000","last_peered":"2024-09-19T21:38:03.972414+0000","last_clean":"2024-09-19T21:38:03.972414+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:03.972414+0000","last_undegraded":"2024-09-19T21:38:03.972414+0000","last_fullsized":"2024-09-19T21:38:03.972414+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7034,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.974149+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:03.974149+0000","last_peered":"2024-09-19T21:38:03.974149+0000","last_clean":"2024-09-19T21:38:03.974149+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:03.974149+0000","last_undegraded":"2024-09-19T21:38:03.974149+0000","last_fullsized":"2024-09-19T21:38:03.974149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10396,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795908+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:04.795908+0000","last_peered":"2024-09-19T21:38:04.795908+0000","last_clean":"2024-09-19T21:38:04.795908+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:04.795908+0000","last_undegraded":"2024-09-19T21:38:04.795908+0000","last_fullsized":"2024-09-19T21:38:04.795908+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796176+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:04.796176+0000","last_peered":"2024-09-19T21:38:04.796176+0000","last_clean":"2024-09-19T21:38:04.796176+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:04.796176+0000","last_undegraded":"2024-09-19T21:38:04.796176+0000","last_fullsized":"2024-09-19T21:38:04.796176+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9034,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793259+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:04.793259+0000","last_peered":"2024-09-19T21:38:04.793259+0000","last_clean":"2024-09-19T21:38:04.793259+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:04.793259+0000","last_undegraded":"2024-09-19T21:38:04.793259+0000","last_fullsized":"2024-09-19T21:38:04.793259+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793436+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:04.793436+0000","last_peered":"2024-09-19T21:38:04.793436+0000","last_clean":"2024-09-19T21:38:04.793436+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:04.793436+0000","last_undegraded":"2024-09-19T21:38:04.793436+0000","last_fullsized":"2024-09-19T21:38:04.793436+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9090,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793496+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:04.793496+0000","last_peered":"2024-09-19T21:38:04.793496+0000","last_clean":"2024-09-19T21:38:04.793496+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:04.793496+0000","last_undegraded":"2024-09-19T21:38:04.793496+0000","last_fullsized":"2024-09-19T21:38:04.793496+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":202,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796031+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:04.796031+0000","last_peered":"2024-09-19T21:38:04.796031+0000","last_clean":"2024-09-19T21:38:04.796031+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:04.796031+0000","last_undegraded":"2024-09-19T21:38:04.796031+0000","last_fullsized":"2024-09-19T21:38:04.796031+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13379,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794209+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.794209+0000","last_peered":"2024-09-19T21:38:04.794209+0000","last_clean":"2024-09-19T21:38:04.794209+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.794209+0000","last_undegraded":"2024-09-19T21:38:04.794209+0000","last_fullsized":"2024-09-19T21:38:04.794209+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":190,"state":"peering","last_fresh":"2024-09-19T21:38:04.990617+0000","last_change":"2024-09-19T21:37:05.846511+0000","last_active":"2024-09-19T21:37:05.846255+0000","last_peered":"2024-09-19T21:37:04.841482+0000","last_clean":"2024-09-19T21:37:04.841482+0000","last_became_active":"2024-09-19T21:36:32.030278+0000","last_became_peered":"2024-09-19T21:36:32.030278+0000","last_unstale":"2024-09-19T21:38:04.990617+0000","last_undegraded":"2024-09-19T21:38:04.990617+0000","last_fullsized":"2024-09-19T21:38:04.990617+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5695,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972153+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:03.972153+0000","last_peered":"2024-09-19T21:38:03.972153+0000","last_clean":"2024-09-19T21:38:03.972153+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:03.972153+0000","last_undegraded":"2024-09-19T21:38:03.972153+0000","last_fullsized":"2024-09-19T21:38:03.972153+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10099,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972983+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:03.972983+0000","last_peered":"2024-09-19T21:38:03.972983+0000","last_clean":"2024-09-19T21:38:03.972983+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:03.972983+0000","last_undegraded":"2024-09-19T21:38:03.972983+0000","last_fullsized":"2024-09-19T21:38:03.972983+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":234,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972272+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:03.972272+0000","last_peered":"2024-09-19T21:38:03.972272+0000","last_clean":"2024-09-19T21:38:03.972272+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:03.972272+0000","last_undegraded":"2024-09-19T21:38:03.972272+0000","last_fullsized":"2024-09-19T21:38:03.972272+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12092,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794181+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.794181+0000","last_peered":"2024-09-19T21:38:04.794181+0000","last_clean":"2024-09-19T21:38:04.794181+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.794181+0000","last_undegraded":"2024-09-19T21:38:04.794181+0000","last_fullsized":"2024-09-19T21:38:04.794181+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9237,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793212+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:04.793212+0000","last_peered":"2024-09-19T21:38:04.793212+0000","last_clean":"2024-09-19T21:38:04.793212+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:04.793212+0000","last_undegraded":"2024-09-19T21:38:04.793212+0000","last_fullsized":"2024-09-19T21:38:04.793212+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.973469+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:03.973469+0000","last_peered":"2024-09-19T21:38:03.973469+0000","last_clean":"2024-09-19T21:38:03.973469+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:03.973469+0000","last_undegraded":"2024-09-19T21:38:03.973469+0000","last_fullsized":"2024-09-19T21:38:03.973469+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":211,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.796057+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:04.796057+0000","last_peered":"2024-09-19T21:38:04.796057+0000","last_clean":"2024-09-19T21:38:04.796057+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:04.796057+0000","last_undegraded":"2024-09-19T21:38:04.796057+0000","last_fullsized":"2024-09-19T21:38:04.796057+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":418,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795727+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:04.795727+0000","last_peered":"2024-09-19T21:38:04.795727+0000","last_clean":"2024-09-19T21:38:04.795727+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:04.795727+0000","last_undegraded":"2024-09-19T21:38:04.795727+0000","last_fullsized":"2024-09-19T21:38:04.795727+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15962,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.793157+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:04.793157+0000","last_peered":"2024-09-19T21:38:04.793157+0000","last_clean":"2024-09-19T21:38:04.793157+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:04.793157+0000","last_undegraded":"2024-09-19T21:38:04.793157+0000","last_fullsized":"2024-09-19T21:38:04.793157+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5835,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972035+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:03.972035+0000","last_peered":"2024-09-19T21:38:03.972035+0000","last_clean":"2024-09-19T21:38:03.972035+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:03.972035+0000","last_undegraded":"2024-09-19T21:38:03.972035+0000","last_fullsized":"2024-09-19T21:38:03.972035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9163,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972162+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:03.972162+0000","last_peered":"2024-09-19T21:38:03.972162+0000","last_clean":"2024-09-19T21:38:03.972162+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:03.972162+0000","last_undegraded":"2024-09-19T21:38:03.972162+0000","last_fullsized":"2024-09-19T21:38:03.972162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4050,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.973382+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:03.973382+0000","last_peered":"2024-09-19T21:38:03.973382+0000","last_clean":"2024-09-19T21:38:03.973382+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:03.973382+0000","last_undegraded":"2024-09-19T21:38:03.973382+0000","last_fullsized":"2024-09-19T21:38:03.973382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9705,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795984+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:04.795984+0000","last_peered":"2024-09-19T21:38:04.795984+0000","last_clean":"2024-09-19T21:38:04.795984+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:04.795984+0000","last_undegraded":"2024-09-19T21:38:04.795984+0000","last_fullsized":"2024-09-19T21:38:04.795984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6431,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972573+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:03.972573+0000","last_peered":"2024-09-19T21:38:03.972573+0000","last_clean":"2024-09-19T21:38:03.972573+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:03.972573+0000","last_undegraded":"2024-09-19T21:38:03.972573+0000","last_fullsized":"2024-09-19T21:38:03.972573+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5591,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.972984+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:03.972984+0000","last_peered":"2024-09-19T21:38:03.972984+0000","last_clean":"2024-09-19T21:38:03.972984+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:03.972984+0000","last_undegraded":"2024-09-19T21:38:03.972984+0000","last_fullsized":"2024-09-19T21:38:03.972984+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8791,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795727+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:04.795727+0000","last_peered":"2024-09-19T21:38:04.795727+0000","last_clean":"2024-09-19T21:38:04.795727+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:04.795727+0000","last_undegraded":"2024-09-19T21:38:04.795727+0000","last_fullsized":"2024-09-19T21:38:04.795727+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:03.971758+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:03.971758+0000","last_peered":"2024-09-19T21:38:03.971758+0000","last_clean":"2024-09-19T21:38:03.971758+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:03.971758+0000","last_undegraded":"2024-09-19T21:38:03.971758+0000","last_fullsized":"2024-09-19T21:38:03.971758+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7485,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.794289+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.794289+0000","last_peered":"2024-09-19T21:38:04.794289+0000","last_clean":"2024-09-19T21:38:04.794289+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.794289+0000","last_undegraded":"2024-09-19T21:38:04.794289+0000","last_fullsized":"2024-09-19T21:38:04.794289+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7910,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.795846+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:04.795846+0000","last_peered":"2024-09-19T21:38:04.795846+0000","last_clean":"2024-09-19T21:38:04.795846+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:04.795846+0000","last_undegraded":"2024-09-19T21:38:04.795846+0000","last_fullsized":"2024-09-19T21:38:04.795846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33928,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337912,"statfs":{"total":96636764160,"available":96602021888,"internally_reserved":0,"allocated":2871296,"data_stored":1538466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.997}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.93700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55472,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316368,"statfs":{"total":96636764160,"available":96579960832,"internally_reserved":0,"allocated":2265088,"data_stored":943128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.0189999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.96399999999999997}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":1.179}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.109},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2273280,"data_stored":948079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.89800000000000002}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.747}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.873}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47876,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323964,"statfs":{"total":96636764160,"available":96587739136,"internally_reserved":0,"allocated":2871296,"data_stored":1538466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2277376,"data_stored":948098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.55900000000000005}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.82999999999999996}]}]},{"osd":3,"up_from":12,"seq":51539607603,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.3009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:08.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T21:38:08.851 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:08.852 INFO:teuthology.orchestra.run.smithi012.stdout:{"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive, 1 pg peering","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":245,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":191,"num_osds":6,"num_up_osds":6,"osd_up_since":1726781652,"num_in_osds":6,"osd_in_since":1726781867,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":329,"data_bytes":590406,"bytes_used":315236352,"bytes_avail":579505348608,"bytes_total":579820584960,"inactive_pgs_ratio":0.024390242993831635,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-19T21:35:14.812992+0000","services":{}},"progress_events":{}} 2024-09-19T21:38:08.866 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-19T21:38:08.866 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:09.187 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:09.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-19T21:38:08.863621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-98,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.909415"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.026},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.62},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92400000000000004}]}]},{"osd":3,"up_from":12,"seq":51539607603,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.3009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:12.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:12.520 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:12.523 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:12.536 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-19T21:38:10.866817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.899295"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.026},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.62},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92400000000000004}]}]},{"osd":3,"up_from":12,"seq":51539607603,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.754},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.3009999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:12.539 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-19T21:38:12.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-19T21:38:12.869 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T21:38:12.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T21:38:12.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:13.187 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:13.203 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T21:38:12.867294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004909"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.026},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.62},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92400000000000004}]}]},{"osd":3,"up_from":12,"seq":51539607604,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.85999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:13.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:13.516 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:13.519 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:13.531 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T21:38:12.867294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004909"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.026},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.62},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92400000000000004}]}]},{"osd":3,"up_from":12,"seq":51539607604,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.85999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:13.533 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T21:38:13.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:13.850 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:13.852 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:13.865 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-19T21:38:12.867294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004909"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.746},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.026},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.498}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.84999999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.98999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.505},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":1.03},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.91700000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.006},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.5},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.62},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92400000000000004}]}]},{"osd":3,"up_from":12,"seq":51539607604,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.85999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:13.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T21:38:14.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:14.181 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":191,"fsid":"5c98f4ba-835c-4398-91b1-e251a78b0f2b","created":"2024-09-19T21:34:03.067364+0000","modified":"2024-09-19T21:38:05.983809+0000","last_up_change":"2024-09-19T21:34:12.817626+0000","last_in_change":"2024-09-19T21:37:47.840357+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T21:34:14.862212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T21:34:19.884271+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-19T21:34:28.955849+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":85,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"66daac4d-0aeb-4d6e-8f7d-05d76e8fdbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6810","nonce":226723117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6814","nonce":226723117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6821","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6823","nonce":226723117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":226723117},{"type":"v1","addr":"172.21.15.12:6818","nonce":226723117}]},"public_addr":"172.21.15.12:6810/226723117","cluster_addr":"172.21.15.12:6814/226723117","heartbeat_back_addr":"172.21.15.12:6823/226723117","heartbeat_front_addr":"172.21.15.12:6818/226723117","state":["exists","up"]},{"osd":1,"uuid":"4d18ba5b-2e50-4a2b-9ca7-e4d046c654b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6801","nonce":1690053836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6803","nonce":1690053836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6807","nonce":1690053836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1690053836},{"type":"v1","addr":"172.21.15.12:6805","nonce":1690053836}]},"public_addr":"172.21.15.12:6801/1690053836","cluster_addr":"172.21.15.12:6803/1690053836","heartbeat_back_addr":"172.21.15.12:6807/1690053836","heartbeat_front_addr":"172.21.15.12:6805/1690053836","state":["exists","up"]},{"osd":2,"uuid":"585d0e6c-0092-4cdb-ae1a-6260630475d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6811","nonce":3902303328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6813","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6815","nonce":3902303328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6822","nonce":3902303328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6817","nonce":3902303328},{"type":"v1","addr":"172.21.15.12:6819","nonce":3902303328}]},"public_addr":"172.21.15.12:6811/3902303328","cluster_addr":"172.21.15.12:6815/3902303328","heartbeat_back_addr":"172.21.15.12:6822/3902303328","heartbeat_front_addr":"172.21.15.12:6819/3902303328","state":["exists","up"]},{"osd":3,"uuid":"41e181a0-e2f9-491b-9b3d-80f5e108577c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6801","nonce":2014863703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6803","nonce":2014863703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6809","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6811","nonce":2014863703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2014863703},{"type":"v1","addr":"172.21.15.33:6807","nonce":2014863703}]},"public_addr":"172.21.15.33:6801/2014863703","cluster_addr":"172.21.15.33:6803/2014863703","heartbeat_back_addr":"172.21.15.33:6811/2014863703","heartbeat_front_addr":"172.21.15.33:6807/2014863703","state":["exists","up"]},{"osd":4,"uuid":"0c81c5a4-40c7-43cc-8e53-553d71bebe2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6810","nonce":2893455664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6815","nonce":2893455664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6821","nonce":2893455664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2893455664},{"type":"v1","addr":"172.21.15.33:6818","nonce":2893455664}]},"public_addr":"172.21.15.33:6810/2893455664","cluster_addr":"172.21.15.33:6815/2893455664","heartbeat_back_addr":"172.21.15.33:6821/2893455664","heartbeat_front_addr":"172.21.15.33:6818/2893455664","state":["exists","up"]},{"osd":5,"uuid":"c161c258-bee9-41b5-8257-1ba140f79b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6805","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6808","nonce":4275333014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6814","nonce":4275333014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6823","nonce":4275333014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":4275333014},{"type":"v1","addr":"172.21.15.33:6819","nonce":4275333014}]},"public_addr":"172.21.15.33:6808/4275333014","cluster_addr":"172.21.15.33:6814/4275333014","heartbeat_back_addr":"172.21.15.33:6823/4275333014","heartbeat_front_addr":"172.21.15.33:6819/4275333014","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.115755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.300024+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.859187+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:09.118947+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.197978+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T21:34:10.229555+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.4","osds":[5,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T21:38:15.197 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T21:38:15.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:15.356 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:15.356 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:15.356 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:15.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T21:38:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T21:38:15.703 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T21:38:15.703 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:15.852 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:15.853 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:15.853 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:15.865 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T21:38:16.179 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T21:38:16.190 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T21:38:16.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:16.348 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:16.348 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:16.348 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:16.362 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T21:38:16.666 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T21:38:16.676 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T21:38:16.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:16.827 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:16.827 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:16.827 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:16.840 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T21:38:17.150 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T21:38:17.161 INFO:tasks.ceph:Scrubbing osd.4 2024-09-19T21:38:17.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:17.316 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:17.316 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:17.316 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:17.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-19T21:38:17.631 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 4 to deep-scrub 2024-09-19T21:38:17.642 INFO:tasks.ceph:Scrubbing osd.5 2024-09-19T21:38:17.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-19T21:38:17.792 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-09-19T21:38:17.792 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T21:38:17.792 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-09-19T21:38:17.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-19T21:38:18.125 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 5 to deep-scrub 2024-09-19T21:38:18.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:18.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:18.449 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:18.464 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-19T21:38:16.868640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18530,"ondisk_log_size":18530,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":307908,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":293238,"kb_avail":565923132,"statfs":{"total":579820584960,"available":579505287168,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":300276390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.875866"},"pg_stats":[{"pgid":"3.1f","version":"172'539","reported_seq":4856,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990573+0000","last_change":"2024-09-19T21:37:47.262192+0000","last_active":"2024-09-19T21:38:04.990573+0000","last_peered":"2024-09-19T21:38:04.990573+0000","last_clean":"2024-09-19T21:38:04.990573+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:04.990573+0000","last_undegraded":"2024-09-19T21:38:04.990573+0000","last_fullsized":"2024-09-19T21:38:04.990573+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:02.477804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"170'281","reported_seq":2691,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987306+0000","last_change":"2024-09-19T21:37:47.149634+0000","last_active":"2024-09-19T21:38:04.987306+0000","last_peered":"2024-09-19T21:38:04.987306+0000","last_clean":"2024-09-19T21:38:04.987306+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:04.987306+0000","last_undegraded":"2024-09-19T21:38:04.987306+0000","last_fullsized":"2024-09-19T21:38:04.987306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:57:03.554019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10523,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989173+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:04.989173+0000","last_peered":"2024-09-19T21:38:04.989173+0000","last_clean":"2024-09-19T21:38:04.989173+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:04.989173+0000","last_undegraded":"2024-09-19T21:38:04.989173+0000","last_fullsized":"2024-09-19T21:38:04.989173+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:10:40.571828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16709,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296138+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.296138+0000","last_peered":"2024-09-19T21:38:05.296138+0000","last_clean":"2024-09-19T21:38:05.296138+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.296138+0000","last_undegraded":"2024-09-19T21:38:05.296138+0000","last_fullsized":"2024-09-19T21:38:05.296138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:53:09.102382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"171'421","reported_seq":4608,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986840+0000","last_change":"2024-09-19T21:37:47.148190+0000","last_active":"2024-09-19T21:38:04.986840+0000","last_peered":"2024-09-19T21:38:04.986840+0000","last_clean":"2024-09-19T21:38:04.986840+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:04.986840+0000","last_undegraded":"2024-09-19T21:38:04.986840+0000","last_fullsized":"2024-09-19T21:38:04.986840+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:48:02.216609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"171'1016","reported_seq":9302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295929+0000","last_change":"2024-09-19T21:37:47.147389+0000","last_active":"2024-09-19T21:38:05.295929+0000","last_peered":"2024-09-19T21:38:05.295929+0000","last_clean":"2024-09-19T21:38:05.295929+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:05.295929+0000","last_undegraded":"2024-09-19T21:38:05.295929+0000","last_fullsized":"2024-09-19T21:38:05.295929+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1016,"log_dups_size":0,"ondisk_log_size":1016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:15.446336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"171'435","reported_seq":14871,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294224+0000","last_change":"2024-09-19T21:37:52.859468+0000","last_active":"2024-09-19T21:38:05.294224+0000","last_peered":"2024-09-19T21:38:05.294224+0000","last_clean":"2024-09-19T21:38:05.294224+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:05.294224+0000","last_undegraded":"2024-09-19T21:38:05.294224+0000","last_fullsized":"2024-09-19T21:38:05.294224+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:37:32.599163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'341","reported_seq":10940,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986726+0000","last_change":"2024-09-19T21:37:51.871745+0000","last_active":"2024-09-19T21:38:04.986726+0000","last_peered":"2024-09-19T21:38:04.986726+0000","last_clean":"2024-09-19T21:38:04.986726+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:04.986726+0000","last_undegraded":"2024-09-19T21:38:04.986726+0000","last_fullsized":"2024-09-19T21:38:04.986726+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:55:23.339964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"171'328","reported_seq":6443,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990526+0000","last_change":"2024-09-19T21:37:47.259643+0000","last_active":"2024-09-19T21:38:04.990526+0000","last_peered":"2024-09-19T21:38:04.990526+0000","last_clean":"2024-09-19T21:38:04.990526+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:04.990526+0000","last_undegraded":"2024-09-19T21:38:04.990526+0000","last_fullsized":"2024-09-19T21:38:04.990526+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:23:49.768260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989194+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:04.989194+0000","last_peered":"2024-09-19T21:38:04.989194+0000","last_clean":"2024-09-19T21:38:04.989194+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:04.989194+0000","last_undegraded":"2024-09-19T21:38:04.989194+0000","last_fullsized":"2024-09-19T21:38:04.989194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:54:44.397008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"170'409","reported_seq":7035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988074+0000","last_change":"2024-09-19T21:37:47.158414+0000","last_active":"2024-09-19T21:38:04.988074+0000","last_peered":"2024-09-19T21:38:04.988074+0000","last_clean":"2024-09-19T21:38:04.988074+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:04.988074+0000","last_undegraded":"2024-09-19T21:38:04.988074+0000","last_fullsized":"2024-09-19T21:38:04.988074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:47:38.149416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10397,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296119+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.296119+0000","last_peered":"2024-09-19T21:38:05.296119+0000","last_clean":"2024-09-19T21:38:05.296119+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.296119+0000","last_undegraded":"2024-09-19T21:38:05.296119+0000","last_fullsized":"2024-09-19T21:38:05.296119+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:34:25.993005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296163+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.296163+0000","last_peered":"2024-09-19T21:38:05.296163+0000","last_clean":"2024-09-19T21:38:05.296163+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.296163+0000","last_undegraded":"2024-09-19T21:38:05.296163+0000","last_fullsized":"2024-09-19T21:38:05.296163+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:21:13.796308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"172'598","reported_seq":9035,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294261+0000","last_change":"2024-09-19T21:37:51.873171+0000","last_active":"2024-09-19T21:38:05.294261+0000","last_peered":"2024-09-19T21:38:05.294261+0000","last_clean":"2024-09-19T21:38:05.294261+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:05.294261+0000","last_undegraded":"2024-09-19T21:38:05.294261+0000","last_fullsized":"2024-09-19T21:38:05.294261+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:51:33.173538+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294172+0000","last_change":"2024-09-19T21:37:48.856541+0000","last_active":"2024-09-19T21:38:05.294172+0000","last_peered":"2024-09-19T21:38:05.294172+0000","last_clean":"2024-09-19T21:38:05.294172+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:05.294172+0000","last_undegraded":"2024-09-19T21:38:05.294172+0000","last_fullsized":"2024-09-19T21:38:05.294172+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:38:26.415665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9091,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294176+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:05.294176+0000","last_peered":"2024-09-19T21:38:05.294176+0000","last_clean":"2024-09-19T21:38:05.294176+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:05.294176+0000","last_undegraded":"2024-09-19T21:38:05.294176+0000","last_fullsized":"2024-09-19T21:38:05.294176+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:06:44.395439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296170+0000","last_change":"2024-09-19T21:37:48.860679+0000","last_active":"2024-09-19T21:38:05.296170+0000","last_peered":"2024-09-19T21:38:05.296170+0000","last_clean":"2024-09-19T21:38:05.296170+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:05.296170+0000","last_undegraded":"2024-09-19T21:38:05.296170+0000","last_fullsized":"2024-09-19T21:38:05.296170+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:06:20.868121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"171'533","reported_seq":13380,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990400+0000","last_change":"2024-09-19T21:37:47.259578+0000","last_active":"2024-09-19T21:38:04.990400+0000","last_peered":"2024-09-19T21:38:04.990400+0000","last_clean":"2024-09-19T21:38:04.990400+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:04.990400+0000","last_undegraded":"2024-09-19T21:38:04.990400+0000","last_fullsized":"2024-09-19T21:38:04.990400+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:28:28.552517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001646+0000","last_change":"2024-09-19T21:38:06.001646+0000","last_active":"2024-09-19T21:38:06.001646+0000","last_peered":"2024-09-19T21:38:06.001646+0000","last_clean":"2024-09-19T21:38:06.001646+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:06.001646+0000","last_undegraded":"2024-09-19T21:38:06.001646+0000","last_fullsized":"2024-09-19T21:38:06.001646+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:03:03.939520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"170'882","reported_seq":5696,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989034+0000","last_change":"2024-09-19T21:37:47.149010+0000","last_active":"2024-09-19T21:38:04.989034+0000","last_peered":"2024-09-19T21:38:04.989034+0000","last_clean":"2024-09-19T21:38:04.989034+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:04.989034+0000","last_undegraded":"2024-09-19T21:38:04.989034+0000","last_fullsized":"2024-09-19T21:38:04.989034+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:21:55.198151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10100,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987476+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:04.987476+0000","last_peered":"2024-09-19T21:38:04.987476+0000","last_clean":"2024-09-19T21:38:04.987476+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:04.987476+0000","last_undegraded":"2024-09-19T21:38:04.987476+0000","last_fullsized":"2024-09-19T21:38:04.987476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:59:50.218593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":235,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987280+0000","last_change":"2024-09-19T21:37:48.861605+0000","last_active":"2024-09-19T21:38:04.987280+0000","last_peered":"2024-09-19T21:38:04.987280+0000","last_clean":"2024-09-19T21:38:04.987280+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:04.987280+0000","last_undegraded":"2024-09-19T21:38:04.987280+0000","last_fullsized":"2024-09-19T21:38:04.987280+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:45:10.286686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"170'570","reported_seq":12093,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990449+0000","last_change":"2024-09-19T21:37:47.261209+0000","last_active":"2024-09-19T21:38:04.990449+0000","last_peered":"2024-09-19T21:38:04.990449+0000","last_clean":"2024-09-19T21:38:04.990449+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:04.990449+0000","last_undegraded":"2024-09-19T21:38:04.990449+0000","last_fullsized":"2024-09-19T21:38:04.990449+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:58:12.213805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":203,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986874+0000","last_change":"2024-09-19T21:37:06.858067+0000","last_active":"2024-09-19T21:38:04.986874+0000","last_peered":"2024-09-19T21:38:04.986874+0000","last_clean":"2024-09-19T21:38:04.986874+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:04.986874+0000","last_undegraded":"2024-09-19T21:38:04.986874+0000","last_fullsized":"2024-09-19T21:38:04.986874+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:30:15.160590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"172'772","reported_seq":9238,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294084+0000","last_change":"2024-09-19T21:37:52.862684+0000","last_active":"2024-09-19T21:38:05.294084+0000","last_peered":"2024-09-19T21:38:05.294084+0000","last_clean":"2024-09-19T21:38:05.294084+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:05.294084+0000","last_undegraded":"2024-09-19T21:38:05.294084+0000","last_fullsized":"2024-09-19T21:38:05.294084+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:58:08.432801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988675+0000","last_change":"2024-09-19T21:37:06.861110+0000","last_active":"2024-09-19T21:38:04.988675+0000","last_peered":"2024-09-19T21:38:04.988675+0000","last_clean":"2024-09-19T21:38:04.988675+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:04.988675+0000","last_undegraded":"2024-09-19T21:38:04.988675+0000","last_fullsized":"2024-09-19T21:38:04.988675+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:00:23.362563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"171'496","reported_seq":12901,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986820+0000","last_change":"2024-09-19T21:37:47.147806+0000","last_active":"2024-09-19T21:38:04.986820+0000","last_peered":"2024-09-19T21:38:04.986820+0000","last_clean":"2024-09-19T21:38:04.986820+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:04.986820+0000","last_undegraded":"2024-09-19T21:38:04.986820+0000","last_fullsized":"2024-09-19T21:38:04.986820+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:11:16.686792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":212,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296075+0000","last_change":"2024-09-19T21:36:32.037272+0000","last_active":"2024-09-19T21:38:05.296075+0000","last_peered":"2024-09-19T21:38:05.296075+0000","last_clean":"2024-09-19T21:38:05.296075+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:05.296075+0000","last_undegraded":"2024-09-19T21:38:05.296075+0000","last_fullsized":"2024-09-19T21:38:05.296075+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:34:04.077130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":419,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295817+0000","last_change":"2024-09-19T21:37:11.904940+0000","last_active":"2024-09-19T21:38:05.295817+0000","last_peered":"2024-09-19T21:38:05.295817+0000","last_clean":"2024-09-19T21:38:05.295817+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:05.295817+0000","last_undegraded":"2024-09-19T21:38:05.295817+0000","last_fullsized":"2024-09-19T21:38:05.295817+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:14.912468+0000","last_clean_scrub_stamp":"2024-09-19T21:34:14.912468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:16:04.347513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"172'547","reported_seq":15963,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.294026+0000","last_change":"2024-09-19T21:37:51.874567+0000","last_active":"2024-09-19T21:38:05.294026+0000","last_peered":"2024-09-19T21:38:05.294026+0000","last_clean":"2024-09-19T21:38:05.294026+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:05.294026+0000","last_undegraded":"2024-09-19T21:38:05.294026+0000","last_fullsized":"2024-09-19T21:38:05.294026+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:53:35.264082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5836,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.986996+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:04.986996+0000","last_peered":"2024-09-19T21:38:04.986996+0000","last_clean":"2024-09-19T21:38:04.986996+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:04.986996+0000","last_undegraded":"2024-09-19T21:38:04.986996+0000","last_fullsized":"2024-09-19T21:38:04.986996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:58:30.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'667","reported_seq":9164,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987017+0000","last_change":"2024-09-19T21:37:59.934041+0000","last_active":"2024-09-19T21:38:04.987017+0000","last_peered":"2024-09-19T21:38:04.987017+0000","last_clean":"2024-09-19T21:38:04.987017+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:04.987017+0000","last_undegraded":"2024-09-19T21:38:04.987017+0000","last_fullsized":"2024-09-19T21:38:04.987017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:35:52.628054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"171'325","reported_seq":2613,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987059+0000","last_change":"2024-09-19T21:37:47.148809+0000","last_active":"2024-09-19T21:38:04.987059+0000","last_peered":"2024-09-19T21:38:04.987059+0000","last_clean":"2024-09-19T21:38:04.987059+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:04.987059+0000","last_undegraded":"2024-09-19T21:38:04.987059+0000","last_fullsized":"2024-09-19T21:38:04.987059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:35:05.613158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4051,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.987590+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:04.987590+0000","last_peered":"2024-09-19T21:38:04.987590+0000","last_clean":"2024-09-19T21:38:04.987590+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:04.987590+0000","last_undegraded":"2024-09-19T21:38:04.987590+0000","last_fullsized":"2024-09-19T21:38:04.987590+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:03:15.265682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'500","reported_seq":9706,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.296310+0000","last_change":"2024-09-19T21:37:47.147470+0000","last_active":"2024-09-19T21:38:05.296310+0000","last_peered":"2024-09-19T21:38:05.296310+0000","last_clean":"2024-09-19T21:38:05.296310+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:05.296310+0000","last_undegraded":"2024-09-19T21:38:05.296310+0000","last_fullsized":"2024-09-19T21:38:05.296310+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:46:47.196998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"171'588","reported_seq":6432,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.989212+0000","last_change":"2024-09-19T21:37:47.152539+0000","last_active":"2024-09-19T21:38:04.989212+0000","last_peered":"2024-09-19T21:38:04.989212+0000","last_clean":"2024-09-19T21:38:04.989212+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:04.989212+0000","last_undegraded":"2024-09-19T21:38:04.989212+0000","last_fullsized":"2024-09-19T21:38:04.989212+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:10:49.952314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"171'858","reported_seq":5592,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988421+0000","last_change":"2024-09-19T21:37:59.934302+0000","last_active":"2024-09-19T21:38:04.988421+0000","last_peered":"2024-09-19T21:38:04.988421+0000","last_clean":"2024-09-19T21:38:04.988421+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:04.988421+0000","last_undegraded":"2024-09-19T21:38:04.988421+0000","last_fullsized":"2024-09-19T21:38:04.988421+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:19:06.121120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"170'606","reported_seq":8792,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295750+0000","last_change":"2024-09-19T21:37:47.146830+0000","last_active":"2024-09-19T21:38:05.295750+0000","last_peered":"2024-09-19T21:38:05.295750+0000","last_clean":"2024-09-19T21:38:05.295750+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:05.295750+0000","last_undegraded":"2024-09-19T21:38:05.295750+0000","last_fullsized":"2024-09-19T21:38:05.295750+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:49:31.899027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"171'435","reported_seq":4250,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.988960+0000","last_change":"2024-09-19T21:37:47.153091+0000","last_active":"2024-09-19T21:38:04.988960+0000","last_peered":"2024-09-19T21:38:04.988960+0000","last_clean":"2024-09-19T21:38:04.988960+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:04.988960+0000","last_undegraded":"2024-09-19T21:38:04.988960+0000","last_fullsized":"2024-09-19T21:38:04.988960+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:51:18.485220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"170'297","reported_seq":7486,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:04.990395+0000","last_change":"2024-09-19T21:37:47.261115+0000","last_active":"2024-09-19T21:38:04.990395+0000","last_peered":"2024-09-19T21:38:04.990395+0000","last_clean":"2024-09-19T21:38:04.990395+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:04.990395+0000","last_undegraded":"2024-09-19T21:38:04.990395+0000","last_fullsized":"2024-09-19T21:38:04.990395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:33:29.355561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"172'416","reported_seq":7911,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.295904+0000","last_change":"2024-09-19T21:37:47.146746+0000","last_active":"2024-09-19T21:38:05.295904+0000","last_peered":"2024-09-19T21:38:05.295904+0000","last_clean":"2024-09-19T21:38:05.295904+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:05.295904+0000","last_undegraded":"2024-09-19T21:38:05.295904+0000","last_fullsized":"2024-09-19T21:38:05.295904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:51:05.058419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18410,"ondisk_log_size":18410,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33932,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337908,"statfs":{"total":96636764160,"available":96602017792,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.05,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.96299999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.79800000000000004,"15min":0.83399999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.41299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78400000000000003,"15min":0.80900000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.432,"5min":0.379,"15min":0.379},"max":{"1min":1.2070000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.84599999999999997,"15min":0.85799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.627}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89500000000000002,"15min":0.93200000000000005},"min":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1850000000000001,"5min":1.381,"15min":1.381},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55504,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316336,"statfs":{"total":96636764160,"available":96579928064,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.69499999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73699999999999999,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.0309999999999999,"5min":1.169,"15min":1.169},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.67900000000000005,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.089,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73499999999999999,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.94499999999999995,"5min":1.048,"15min":1.048},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.94099999999999995,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.56200000000000006,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.94399999999999995,"5min":0.996,"15min":0.996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.50700000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:37:18 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79400000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.374,"15min":0.374},"max":{"1min":0.96999999999999997,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85499999999999998,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0329999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.91600000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.03,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.57299999999999995,"15min":0.57899999999999996},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69799999999999995,"5min":1.359,"15min":1.359},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.33500000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.78500000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0800000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.873}]},{"osd":3,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.51000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83899999999999997,"15min":0.86199999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1759999999999999,"5min":1.921,"15min":1.921},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2070000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.147,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:37:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0049999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.80400000000000005,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.96699999999999997,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.70899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.621,"15min":0.59999999999999998},"min":{"1min":0.29399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.73799999999999999,"15min":0.65700000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":3.742,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.81399999999999995,"15min":0.78200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.448,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.508,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2290000000000001,"5min":1.554,"15min":1.554},"last":0.97899999999999998}]},{"osd":5,"last update":"Thu Sep 19 21:37:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.90000000000000002,"15min":0.85799999999999998},"min":{"1min":0.51300000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.637,"5min":3.637,"15min":3.637},"last":0.94099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55496,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316344,"statfs":{"total":96636764160,"available":96579936256,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.27500000000000002,"5min":0.183,"15min":0.183},"max":{"1min":0.92400000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58199999999999996,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.69099999999999995,"5min":1.333,"15min":1.333},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.73799999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.78200000000000003},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.052,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.48699999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.63,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88500000000000001,"15min":0.89400000000000002},"min":{"1min":0.53200000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.631,"5min":1.718,"15min":1.718},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Sep 19 21:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.141,"5min":1.248,"15min":1.248},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.88700000000000001}]}]},{"osd":3,"up_from":12,"seq":51539607605,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59592,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312248,"statfs":{"total":96636764160,"available":96575741952,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":58718664},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91200000000000003,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.755},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.67600000000000005,"15min":0.70099999999999996},"min":{"1min":0.437,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.91900000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.1060000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0489999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.92300000000000004}]},{"osd":2,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":1.1830000000000001,"15min":0.82899999999999996},"min":{"1min":0.433,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.194,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.254,"15min":0.90000000000000002},"min":{"1min":0.56000000000000005,"5min":0.379,"15min":0.379},"max":{"1min":1.103,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.0760000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.71699999999999997},"min":{"1min":0.61199999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Sep 19 21:37:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.78200000000000003,"15min":0.76400000000000001},"min":{"1min":0.58399999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.077,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.470 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.471 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.472 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T21:34:14.912468+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.473 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:18.474 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T21:38:38.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:38.803 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:38.805 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:38.819 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-19T21:38:36.875189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18790,"ondisk_log_size":18790,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311960,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":297334,"kb_avail":565919080,"statfs":{"total":579820584960,"available":579501137920,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":304470694},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004050"},"pg_stats":[{"pgid":"3.1f","version":"191'545","reported_seq":4874,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.220212+0000","last_change":"2024-09-19T21:38:20.220410+0000","last_active":"2024-09-19T21:38:20.220212+0000","last_peered":"2024-09-19T21:38:20.220212+0000","last_clean":"2024-09-19T21:38:20.220212+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:20.220212+0000","last_undegraded":"2024-09-19T21:38:20.220212+0000","last_fullsized":"2024-09-19T21:38:20.220212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'545","last_scrub_stamp":"2024-09-19T21:38:20.220176+0000","last_deep_scrub":"191'545","last_deep_scrub_stamp":"2024-09-19T21:38:20.220176+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.220176+0000","objects_scrubbed":6,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:30:48.946650+0000","scrub_duration":0.016243944,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"191'295","reported_seq":2718,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.474184+0000","last_change":"2024-09-19T21:38:28.474184+0000","last_active":"2024-09-19T21:38:28.474184+0000","last_peered":"2024-09-19T21:38:28.474184+0000","last_clean":"2024-09-19T21:38:28.474184+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:28.474184+0000","last_undegraded":"2024-09-19T21:38:28.474184+0000","last_fullsized":"2024-09-19T21:38:28.474184+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'295","last_scrub_stamp":"2024-09-19T21:38:28.473931+0000","last_deep_scrub":"191'295","last_deep_scrub_stamp":"2024-09-19T21:38:28.473931+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.473931+0000","objects_scrubbed":14,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:50:21.235837+0000","scrub_duration":0.020161294999999999,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"171'1566","reported_seq":10524,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001432+0000","last_change":"2024-09-19T21:37:47.150406+0000","last_active":"2024-09-19T21:38:06.001432+0000","last_peered":"2024-09-19T21:38:06.001432+0000","last_clean":"2024-09-19T21:38:06.001432+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:06.001432+0000","last_undegraded":"2024-09-19T21:38:06.001432+0000","last_fullsized":"2024-09-19T21:38:06.001432+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1566,"log_dups_size":0,"ondisk_log_size":1566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"172'662","reported_seq":16710,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.994870+0000","last_change":"2024-09-19T21:37:47.148223+0000","last_active":"2024-09-19T21:38:05.994870+0000","last_peered":"2024-09-19T21:38:05.994870+0000","last_clean":"2024-09-19T21:38:05.994870+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:05.994870+0000","last_undegraded":"2024-09-19T21:38:05.994870+0000","last_fullsized":"2024-09-19T21:38:05.994870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"191'430","reported_seq":4629,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.436111+0000","last_change":"2024-09-19T21:38:29.436462+0000","last_active":"2024-09-19T21:38:29.436111+0000","last_peered":"2024-09-19T21:38:29.436111+0000","last_clean":"2024-09-19T21:38:29.436111+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:29.436111+0000","last_undegraded":"2024-09-19T21:38:29.436111+0000","last_fullsized":"2024-09-19T21:38:29.436111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'430","last_scrub_stamp":"2024-09-19T21:38:29.436062+0000","last_deep_scrub":"191'430","last_deep_scrub_stamp":"2024-09-19T21:38:29.436062+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.436062+0000","objects_scrubbed":9,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:11:13.461944+0000","scrub_duration":0.016310802999999999,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"191'1025","reported_seq":9323,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:32.641312+0000","last_change":"2024-09-19T21:38:32.641436+0000","last_active":"2024-09-19T21:38:32.641312+0000","last_peered":"2024-09-19T21:38:32.641312+0000","last_clean":"2024-09-19T21:38:32.641312+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:32.641312+0000","last_undegraded":"2024-09-19T21:38:32.641312+0000","last_fullsized":"2024-09-19T21:38:32.641312+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1025","last_scrub_stamp":"2024-09-19T21:38:32.641289+0000","last_deep_scrub":"191'1025","last_deep_scrub_stamp":"2024-09-19T21:38:32.641289+0000","last_clean_scrub_stamp":"2024-09-19T21:38:32.641289+0000","objects_scrubbed":9,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:42:11.228503+0000","scrub_duration":0.022733077000000001,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"191'442","reported_seq":14890,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:22.110968+0000","last_change":"2024-09-19T21:38:22.111093+0000","last_active":"2024-09-19T21:38:22.110968+0000","last_peered":"2024-09-19T21:38:22.110968+0000","last_clean":"2024-09-19T21:38:22.110968+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:22.110968+0000","last_undegraded":"2024-09-19T21:38:22.110968+0000","last_fullsized":"2024-09-19T21:38:22.110968+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'442","last_scrub_stamp":"2024-09-19T21:38:22.110946+0000","last_deep_scrub":"191'442","last_deep_scrub_stamp":"2024-09-19T21:38:22.110946+0000","last_clean_scrub_stamp":"2024-09-19T21:38:22.110946+0000","objects_scrubbed":7,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:47:57.884771+0000","scrub_duration":0.012774656000000001,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"191'351","reported_seq":10962,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:25.488308+0000","last_change":"2024-09-19T21:38:25.488455+0000","last_active":"2024-09-19T21:38:25.488308+0000","last_peered":"2024-09-19T21:38:25.488308+0000","last_clean":"2024-09-19T21:38:25.488308+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:25.488308+0000","last_undegraded":"2024-09-19T21:38:25.488308+0000","last_fullsized":"2024-09-19T21:38:25.488308+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'351","last_scrub_stamp":"2024-09-19T21:38:25.488287+0000","last_deep_scrub":"191'351","last_deep_scrub_stamp":"2024-09-19T21:38:25.488287+0000","last_clean_scrub_stamp":"2024-09-19T21:38:25.488287+0000","objects_scrubbed":10,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:17:54.432575+0000","scrub_duration":0.01809614,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"191'338","reported_seq":6465,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.365960+0000","last_change":"2024-09-19T21:38:26.366104+0000","last_active":"2024-09-19T21:38:26.365960+0000","last_peered":"2024-09-19T21:38:26.365960+0000","last_clean":"2024-09-19T21:38:26.365960+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:26.365960+0000","last_undegraded":"2024-09-19T21:38:26.365960+0000","last_fullsized":"2024-09-19T21:38:26.365960+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'338","last_scrub_stamp":"2024-09-19T21:38:26.365933+0000","last_deep_scrub":"191'338","last_deep_scrub_stamp":"2024-09-19T21:38:26.365933+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.365933+0000","objects_scrubbed":10,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:05.018045+0000","scrub_duration":0.017908897999999999,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"171'379","reported_seq":11288,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.001300+0000","last_change":"2024-09-19T21:37:47.151836+0000","last_active":"2024-09-19T21:38:06.001300+0000","last_peered":"2024-09-19T21:38:06.001300+0000","last_clean":"2024-09-19T21:38:06.001300+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:06.001300+0000","last_undegraded":"2024-09-19T21:38:06.001300+0000","last_fullsized":"2024-09-19T21:38:06.001300+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"191'420","reported_seq":7058,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.284875+0000","last_change":"2024-09-19T21:38:21.285051+0000","last_active":"2024-09-19T21:38:21.284875+0000","last_peered":"2024-09-19T21:38:21.284875+0000","last_clean":"2024-09-19T21:38:21.284875+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:21.284875+0000","last_undegraded":"2024-09-19T21:38:21.284875+0000","last_fullsized":"2024-09-19T21:38:21.284875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'420","last_scrub_stamp":"2024-09-19T21:38:21.284849+0000","last_deep_scrub":"191'420","last_deep_scrub_stamp":"2024-09-19T21:38:21.284849+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.284849+0000","objects_scrubbed":11,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:14:33.158842+0000","scrub_duration":0.022286953000000002,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'367","reported_seq":10398,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.994765+0000","last_change":"2024-09-19T21:37:59.935478+0000","last_active":"2024-09-19T21:38:05.994765+0000","last_peered":"2024-09-19T21:38:05.994765+0000","last_clean":"2024-09-19T21:38:05.994765+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:05.994765+0000","last_undegraded":"2024-09-19T21:38:05.994765+0000","last_fullsized":"2024-09-19T21:38:05.994765+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:05.995049+0000","last_change":"2024-09-19T21:37:56.902164+0000","last_active":"2024-09-19T21:38:05.995049+0000","last_peered":"2024-09-19T21:38:05.995049+0000","last_clean":"2024-09-19T21:38:05.995049+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:05.995049+0000","last_undegraded":"2024-09-19T21:38:05.995049+0000","last_fullsized":"2024-09-19T21:38:05.995049+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:19.938604+0000","last_clean_scrub_stamp":"2024-09-19T21:34:19.938604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"191'602","reported_seq":9051,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.168524+0000","last_change":"2024-09-19T21:38:29.168706+0000","last_active":"2024-09-19T21:38:29.168524+0000","last_peered":"2024-09-19T21:38:29.168524+0000","last_clean":"2024-09-19T21:38:29.168524+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:29.168524+0000","last_undegraded":"2024-09-19T21:38:29.168524+0000","last_fullsized":"2024-09-19T21:38:29.168524+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'602","last_scrub_stamp":"2024-09-19T21:38:29.168490+0000","last_deep_scrub":"191'602","last_deep_scrub_stamp":"2024-09-19T21:38:29.168490+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.168490+0000","objects_scrubbed":4,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:29:38.267584+0000","scrub_duration":0.012096908999999999,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.150002+0000","last_change":"2024-09-19T21:38:21.150321+0000","last_active":"2024-09-19T21:38:21.150002+0000","last_peered":"2024-09-19T21:38:21.150002+0000","last_clean":"2024-09-19T21:38:21.150002+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:21.150002+0000","last_undegraded":"2024-09-19T21:38:21.150002+0000","last_fullsized":"2024-09-19T21:38:21.150002+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:21.149956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:21.149956+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.149956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:36:46.810807+0000","scrub_duration":0.0034548930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'772","reported_seq":9092,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.394758+0000","last_change":"2024-09-19T21:37:53.870735+0000","last_active":"2024-09-19T21:38:06.394758+0000","last_peered":"2024-09-19T21:38:06.394758+0000","last_clean":"2024-09-19T21:38:06.394758+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:06.394758+0000","last_undegraded":"2024-09-19T21:38:06.394758+0000","last_fullsized":"2024-09-19T21:38:06.394758+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.595490+0000","last_change":"2024-09-19T21:38:21.595490+0000","last_active":"2024-09-19T21:38:21.595490+0000","last_peered":"2024-09-19T21:38:21.595490+0000","last_clean":"2024-09-19T21:38:21.595490+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:21.595490+0000","last_undegraded":"2024-09-19T21:38:21.595490+0000","last_fullsized":"2024-09-19T21:38:21.595490+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:21.595049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:21.595049+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.595049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:08:55.022948+0000","scrub_duration":0.003699894,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"191'543","reported_seq":13402,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:27.408581+0000","last_change":"2024-09-19T21:38:27.408818+0000","last_active":"2024-09-19T21:38:27.408581+0000","last_peered":"2024-09-19T21:38:27.408581+0000","last_clean":"2024-09-19T21:38:27.408581+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:27.408581+0000","last_undegraded":"2024-09-19T21:38:27.408581+0000","last_fullsized":"2024-09-19T21:38:27.408581+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'543","last_scrub_stamp":"2024-09-19T21:38:27.408516+0000","last_deep_scrub":"191'543","last_deep_scrub_stamp":"2024-09-19T21:38:27.408516+0000","last_clean_scrub_stamp":"2024-09-19T21:38:27.408516+0000","objects_scrubbed":10,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:18:42.745523+0000","scrub_duration":0.019168872999999999,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.429345+0000","last_change":"2024-09-19T21:38:29.429734+0000","last_active":"2024-09-19T21:38:29.429345+0000","last_peered":"2024-09-19T21:38:29.429345+0000","last_clean":"2024-09-19T21:38:29.429345+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:29.429345+0000","last_undegraded":"2024-09-19T21:38:29.429345+0000","last_fullsized":"2024-09-19T21:38:29.429345+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:29.429256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:29.429256+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.429256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:02:17.288111+0000","scrub_duration":0.0033198419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"191'893","reported_seq":5719,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:23.351998+0000","last_change":"2024-09-19T21:38:23.352176+0000","last_active":"2024-09-19T21:38:23.351998+0000","last_peered":"2024-09-19T21:38:23.351998+0000","last_clean":"2024-09-19T21:38:23.351998+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:23.351998+0000","last_undegraded":"2024-09-19T21:38:23.351998+0000","last_fullsized":"2024-09-19T21:38:23.351998+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'893","last_scrub_stamp":"2024-09-19T21:38:23.351971+0000","last_deep_scrub":"191'893","last_deep_scrub_stamp":"2024-09-19T21:38:23.351971+0000","last_clean_scrub_stamp":"2024-09-19T21:38:23.351971+0000","objects_scrubbed":11,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:07:24.468656+0000","scrub_duration":0.022058634000000001,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'1029","reported_seq":10101,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.270529+0000","last_change":"2024-09-19T21:37:47.159805+0000","last_active":"2024-09-19T21:38:06.270529+0000","last_peered":"2024-09-19T21:38:06.270529+0000","last_clean":"2024-09-19T21:38:06.270529+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:06.270529+0000","last_undegraded":"2024-09-19T21:38:06.270529+0000","last_fullsized":"2024-09-19T21:38:06.270529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"191'3","reported_seq":249,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:18.243818+0000","last_change":"2024-09-19T21:38:18.243818+0000","last_active":"2024-09-19T21:38:18.243818+0000","last_peered":"2024-09-19T21:38:18.243818+0000","last_clean":"2024-09-19T21:38:18.243818+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:18.243818+0000","last_undegraded":"2024-09-19T21:38:18.243818+0000","last_fullsized":"2024-09-19T21:38:18.243818+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'3","last_scrub_stamp":"2024-09-19T21:38:18.243601+0000","last_deep_scrub":"191'3","last_deep_scrub_stamp":"2024-09-19T21:38:18.243601+0000","last_clean_scrub_stamp":"2024-09-19T21:38:18.243601+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:47.473469+0000","scrub_duration":0.0064841209999999998,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"191'581","reported_seq":12116,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:22.241787+0000","last_change":"2024-09-19T21:38:22.241925+0000","last_active":"2024-09-19T21:38:22.241787+0000","last_peered":"2024-09-19T21:38:22.241787+0000","last_clean":"2024-09-19T21:38:22.241787+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:22.241787+0000","last_undegraded":"2024-09-19T21:38:22.241787+0000","last_fullsized":"2024-09-19T21:38:22.241787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'581","last_scrub_stamp":"2024-09-19T21:38:22.241766+0000","last_deep_scrub":"191'581","last_deep_scrub_stamp":"2024-09-19T21:38:22.241766+0000","last_clean_scrub_stamp":"2024-09-19T21:38:22.241766+0000","objects_scrubbed":11,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:02:43.033390+0000","scrub_duration":0.018496238000000002,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.474067+0000","last_change":"2024-09-19T21:38:20.474067+0000","last_active":"2024-09-19T21:38:20.474067+0000","last_peered":"2024-09-19T21:38:20.474067+0000","last_clean":"2024-09-19T21:38:20.474067+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:20.474067+0000","last_undegraded":"2024-09-19T21:38:20.474067+0000","last_fullsized":"2024-09-19T21:38:20.474067+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:20.473691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:20.473691+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.473691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:48:31.793308+0000","scrub_duration":0.0028180929999999998,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"191'781","reported_seq":9259,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.183841+0000","last_change":"2024-09-19T21:38:28.184044+0000","last_active":"2024-09-19T21:38:28.183841+0000","last_peered":"2024-09-19T21:38:28.183841+0000","last_clean":"2024-09-19T21:38:28.183841+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:28.183841+0000","last_undegraded":"2024-09-19T21:38:28.183841+0000","last_fullsized":"2024-09-19T21:38:28.183841+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'781","last_scrub_stamp":"2024-09-19T21:38:28.183785+0000","last_deep_scrub":"191'781","last_deep_scrub_stamp":"2024-09-19T21:38:28.183785+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.183785+0000","objects_scrubbed":9,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:27:10.956335+0000","scrub_duration":0.018140495,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.476160+0000","last_change":"2024-09-19T21:38:30.476160+0000","last_active":"2024-09-19T21:38:30.476160+0000","last_peered":"2024-09-19T21:38:30.476160+0000","last_clean":"2024-09-19T21:38:30.476160+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:30.476160+0000","last_undegraded":"2024-09-19T21:38:30.476160+0000","last_fullsized":"2024-09-19T21:38:30.476160+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:30.475855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:30.475855+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.475855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:56:03.185387+0000","scrub_duration":0.0032581400000000001,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"191'503","reported_seq":12920,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.387572+0000","last_change":"2024-09-19T21:38:30.387987+0000","last_active":"2024-09-19T21:38:30.387572+0000","last_peered":"2024-09-19T21:38:30.387572+0000","last_clean":"2024-09-19T21:38:30.387572+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:30.387572+0000","last_undegraded":"2024-09-19T21:38:30.387572+0000","last_fullsized":"2024-09-19T21:38:30.387572+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'503","last_scrub_stamp":"2024-09-19T21:38:30.387492+0000","last_deep_scrub":"191'503","last_deep_scrub_stamp":"2024-09-19T21:38:30.387492+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.387492+0000","objects_scrubbed":7,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:17.311069+0000","scrub_duration":0.015325646,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"191'2","reported_seq":226,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:25.579128+0000","last_change":"2024-09-19T21:38:25.579128+0000","last_active":"2024-09-19T21:38:25.579128+0000","last_peered":"2024-09-19T21:38:25.579128+0000","last_clean":"2024-09-19T21:38:25.579128+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:25.579128+0000","last_undegraded":"2024-09-19T21:38:25.579128+0000","last_fullsized":"2024-09-19T21:38:25.579128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'2","last_scrub_stamp":"2024-09-19T21:38:25.578844+0000","last_deep_scrub":"191'2","last_deep_scrub_stamp":"2024-09-19T21:38:25.578844+0000","last_clean_scrub_stamp":"2024-09-19T21:38:25.578844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:12:57.236017+0000","scrub_duration":0.0061060630000000001,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"191'119","reported_seq":434,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.559461+0000","last_change":"2024-09-19T21:38:20.559823+0000","last_active":"2024-09-19T21:38:20.559461+0000","last_peered":"2024-09-19T21:38:20.559461+0000","last_clean":"2024-09-19T21:38:20.559461+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:20.559461+0000","last_undegraded":"2024-09-19T21:38:20.559461+0000","last_fullsized":"2024-09-19T21:38:20.559461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'119","last_scrub_stamp":"2024-09-19T21:38:20.559345+0000","last_deep_scrub":"191'119","last_deep_scrub_stamp":"2024-09-19T21:38:20.559345+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.559345+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:00:37.680058+0000","scrub_duration":0.011967459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"191'559","reported_seq":15987,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:18.122470+0000","last_change":"2024-09-19T21:38:18.122594+0000","last_active":"2024-09-19T21:38:18.122470+0000","last_peered":"2024-09-19T21:38:18.122470+0000","last_clean":"2024-09-19T21:38:18.122470+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:18.122470+0000","last_undegraded":"2024-09-19T21:38:18.122470+0000","last_fullsized":"2024-09-19T21:38:18.122470+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'559","last_scrub_stamp":"2024-09-19T21:38:18.122448+0000","last_deep_scrub":"191'559","last_deep_scrub_stamp":"2024-09-19T21:38:18.122448+0000","last_clean_scrub_stamp":"2024-09-19T21:38:18.122448+0000","objects_scrubbed":12,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:49:13.665052+0000","scrub_duration":0.018890111000000001,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'457","reported_seq":5837,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.269408+0000","last_change":"2024-09-19T21:37:47.157821+0000","last_active":"2024-09-19T21:38:06.269408+0000","last_peered":"2024-09-19T21:38:06.269408+0000","last_clean":"2024-09-19T21:38:06.269408+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:06.269408+0000","last_undegraded":"2024-09-19T21:38:06.269408+0000","last_fullsized":"2024-09-19T21:38:06.269408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"191'682","reported_seq":9191,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:19.226397+0000","last_change":"2024-09-19T21:38:19.226614+0000","last_active":"2024-09-19T21:38:19.226397+0000","last_peered":"2024-09-19T21:38:19.226397+0000","last_clean":"2024-09-19T21:38:19.226397+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:19.226397+0000","last_undegraded":"2024-09-19T21:38:19.226397+0000","last_fullsized":"2024-09-19T21:38:19.226397+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'682","last_scrub_stamp":"2024-09-19T21:38:19.226361+0000","last_deep_scrub":"191'682","last_deep_scrub_stamp":"2024-09-19T21:38:19.226361+0000","last_clean_scrub_stamp":"2024-09-19T21:38:19.226361+0000","objects_scrubbed":15,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:43:36.026665+0000","scrub_duration":0.020747162999999999,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"191'335","reported_seq":2635,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.460435+0000","last_change":"2024-09-19T21:38:26.460755+0000","last_active":"2024-09-19T21:38:26.460435+0000","last_peered":"2024-09-19T21:38:26.460435+0000","last_clean":"2024-09-19T21:38:26.460435+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:26.460435+0000","last_undegraded":"2024-09-19T21:38:26.460435+0000","last_fullsized":"2024-09-19T21:38:26.460435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'335","last_scrub_stamp":"2024-09-19T21:38:26.460374+0000","last_deep_scrub":"191'335","last_deep_scrub_stamp":"2024-09-19T21:38:26.460374+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.460374+0000","objects_scrubbed":10,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:29:19.610517+0000","scrub_duration":0.015993165,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'318","reported_seq":4052,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:06.269567+0000","last_change":"2024-09-19T21:37:47.158642+0000","last_active":"2024-09-19T21:38:06.269567+0000","last_peered":"2024-09-19T21:38:06.269567+0000","last_clean":"2024-09-19T21:38:06.269567+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:06.269567+0000","last_undegraded":"2024-09-19T21:38:06.269567+0000","last_fullsized":"2024-09-19T21:38:06.269567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:34:28.971952+0000","last_clean_scrub_stamp":"2024-09-19T21:34:28.971952+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"191'511","reported_seq":9729,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.645640+0000","last_change":"2024-09-19T21:38:30.645836+0000","last_active":"2024-09-19T21:38:30.645640+0000","last_peered":"2024-09-19T21:38:30.645640+0000","last_clean":"2024-09-19T21:38:30.645640+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:30.645640+0000","last_undegraded":"2024-09-19T21:38:30.645640+0000","last_fullsized":"2024-09-19T21:38:30.645640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'511","last_scrub_stamp":"2024-09-19T21:38:30.645613+0000","last_deep_scrub":"191'511","last_deep_scrub_stamp":"2024-09-19T21:38:30.645613+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.645613+0000","objects_scrubbed":11,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:38:28.104714+0000","scrub_duration":0.020534383,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"191'598","reported_seq":6454,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:24.356790+0000","last_change":"2024-09-19T21:38:24.357021+0000","last_active":"2024-09-19T21:38:24.356790+0000","last_peered":"2024-09-19T21:38:24.356790+0000","last_clean":"2024-09-19T21:38:24.356790+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:24.356790+0000","last_undegraded":"2024-09-19T21:38:24.356790+0000","last_fullsized":"2024-09-19T21:38:24.356790+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'598","last_scrub_stamp":"2024-09-19T21:38:24.356738+0000","last_deep_scrub":"191'598","last_deep_scrub_stamp":"2024-09-19T21:38:24.356738+0000","last_clean_scrub_stamp":"2024-09-19T21:38:24.356738+0000","objects_scrubbed":10,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:23:30.351676+0000","scrub_duration":0.018075013000000001,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"191'870","reported_seq":5616,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.251980+0000","last_change":"2024-09-19T21:38:20.252100+0000","last_active":"2024-09-19T21:38:20.251980+0000","last_peered":"2024-09-19T21:38:20.251980+0000","last_clean":"2024-09-19T21:38:20.251980+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:20.251980+0000","last_undegraded":"2024-09-19T21:38:20.251980+0000","last_fullsized":"2024-09-19T21:38:20.251980+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'870","last_scrub_stamp":"2024-09-19T21:38:20.251961+0000","last_deep_scrub":"191'870","last_deep_scrub_stamp":"2024-09-19T21:38:20.251961+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.251961+0000","objects_scrubbed":12,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:15:33.610802+0000","scrub_duration":0.020965451,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"191'627","reported_seq":8825,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.645022+0000","last_change":"2024-09-19T21:38:26.645246+0000","last_active":"2024-09-19T21:38:26.645022+0000","last_peered":"2024-09-19T21:38:26.645022+0000","last_clean":"2024-09-19T21:38:26.645022+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:26.645022+0000","last_undegraded":"2024-09-19T21:38:26.645022+0000","last_fullsized":"2024-09-19T21:38:26.645022+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'627","last_scrub_stamp":"2024-09-19T21:38:26.644980+0000","last_deep_scrub":"191'627","last_deep_scrub_stamp":"2024-09-19T21:38:26.644980+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.644980+0000","objects_scrubbed":21,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:21:20.941788+0000","scrub_duration":0.026630226999999999,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"191'445","reported_seq":4272,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:17.435267+0000","last_change":"2024-09-19T21:38:17.435419+0000","last_active":"2024-09-19T21:38:17.435267+0000","last_peered":"2024-09-19T21:38:17.435267+0000","last_clean":"2024-09-19T21:38:17.435267+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:17.435267+0000","last_undegraded":"2024-09-19T21:38:17.435267+0000","last_fullsized":"2024-09-19T21:38:17.435267+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'445","last_scrub_stamp":"2024-09-19T21:38:17.435226+0000","last_deep_scrub":"191'445","last_deep_scrub_stamp":"2024-09-19T21:38:17.435226+0000","last_clean_scrub_stamp":"2024-09-19T21:38:17.435226+0000","objects_scrubbed":10,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:11:54.744906+0000","scrub_duration":0.016458093,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"191'309","reported_seq":7510,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:19.182295+0000","last_change":"2024-09-19T21:38:19.182444+0000","last_active":"2024-09-19T21:38:19.182295+0000","last_peered":"2024-09-19T21:38:19.182295+0000","last_clean":"2024-09-19T21:38:19.182295+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:19.182295+0000","last_undegraded":"2024-09-19T21:38:19.182295+0000","last_fullsized":"2024-09-19T21:38:19.182295+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'309","last_scrub_stamp":"2024-09-19T21:38:19.182268+0000","last_deep_scrub":"191'309","last_deep_scrub_stamp":"2024-09-19T21:38:19.182268+0000","last_clean_scrub_stamp":"2024-09-19T21:38:19.182268+0000","objects_scrubbed":12,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:22:24.858075+0000","scrub_duration":0.019684788000000002,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"191'431","reported_seq":7938,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.689136+0000","last_change":"2024-09-19T21:38:28.689414+0000","last_active":"2024-09-19T21:38:28.689136+0000","last_peered":"2024-09-19T21:38:28.689136+0000","last_clean":"2024-09-19T21:38:28.689136+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:28.689136+0000","last_undegraded":"2024-09-19T21:38:28.689136+0000","last_fullsized":"2024-09-19T21:38:28.689136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'431","last_scrub_stamp":"2024-09-19T21:38:28.689090+0000","last_deep_scrub":"191'431","last_deep_scrub_stamp":"2024-09-19T21:38:28.689090+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.689090+0000","objects_scrubbed":15,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:46:34.172771+0000","scrub_duration":0.024207152999999999,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18666,"ondisk_log_size":18666,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33928,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337912,"statfs":{"total":96636764160,"available":96602021888,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69899999999999995,"15min":0.70399999999999996},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0029999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.78700000000000003},"min":{"1min":0.56599999999999995,"5min":0.372,"15min":0.372},"max":{"1min":1.107,"5min":1.375,"15min":1.375},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0529999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.749,"15min":0.76000000000000001},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0109999999999999,"5min":1.232,"15min":1.232},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.016,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.379,"15min":0.379},"max":{"1min":1.028,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.58699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98099999999999998,"5min":2.625,"15min":2.625},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.48599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.1799999999999999,"5min":1.488,"15min":1.488},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.92700000000000005},"min":{"1min":0.69399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0880000000000001,"5min":1.381,"15min":1.381},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.58299999999999996,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.026,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.751,"15min":0.75},"min":{"1min":0.50700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98899999999999999,"5min":1.169,"15min":1.169},"last":0.877},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56399999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.66700000000000004,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.95599999999999996,"5min":1.048,"15min":1.048},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.754,"15min":0.753},"min":{"1min":0.52400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.53900000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88500000000000001,"5min":0.996,"15min":0.996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.0129999999999999,"5min":1.179,"15min":1.179},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.109,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.66400000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.78400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.78000000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.497},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.1080000000000001,"5min":1.359,"15min":1.359},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.249,"15min":0.249},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0720000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82899999999999996,"15min":0.85899999999999999},"min":{"1min":0.65100000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.921,"15min":1.921},"last":0.94699999999999995}]},{"osd":4,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.54700000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.879},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97599999999999998,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.745}]},{"osd":5,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.71699999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.1779999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80700000000000005,"15min":0.79800000000000004},"min":{"1min":0.59899999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.79100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47880,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323960,"statfs":{"total":96636764160,"available":96587735040,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.5},"min":{"1min":0.33300000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.91700000000000004,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82999999999999996,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.42299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":1.081,"15min":1.081},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.749,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"min":{"1min":0.58899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.878},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.57499999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.91700000000000004,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0469999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.69699999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1879999999999999,"5min":1.554,"15min":1.554},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.64900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.071,"5min":1.304,"15min":1.304},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90500000000000003,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":3.637,"15min":3.637},"last":0.748}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.55800000000000005},"min":{"1min":0.25700000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.82899999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.496},{"interface":"front","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78600000000000003,"5min":1.333,"15min":1.333},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73799999999999999,"5min":1.234,"15min":1.234},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.53700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1399999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.86299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0529999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2350000000000001}]},{"osd":4,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74099999999999999,"15min":0.73999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0609999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0109999999999999,"5min":1.718,"15min":1.718},"last":1.294}]},{"osd":5,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.248,"15min":1.248},"last":1.008},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83099999999999996,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.90600000000000003}]}]},{"osd":3,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63680,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308160,"statfs":{"total":96636764160,"available":96571555840,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":62912968},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94099999999999995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.96899999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.72299999999999998,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.876},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1950000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.202,"15min":0.83499999999999996},"min":{"1min":0.48899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.127,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.2969999999999999,"15min":0.91500000000000004},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":3.4500000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":1.0669999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76700000000000002,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.167,"5min":1.492,"15min":1.492},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3009999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.64600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-19T21:34:19.938604+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=19, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.822 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.823 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.823 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.823 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-19T21:34:28.971952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=34, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=21, tm_min=38, tm_sec=14, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T21:38:38.823 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T21:38:58.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T21:38:59.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-19T21:38:59.149 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-19T21:38:59.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-19T21:38:58.881918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":329,"num_object_clones":0,"num_object_copies":658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":321,"num_read":303751,"num_read_kb":1060115,"num_write":18765,"num_write_kb":271609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5658,"num_bytes_recovered":284740150,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18859,"ondisk_log_size":18859,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":311960,"kb_used_data":14568,"kb_used_omap":9,"kb_used_meta":297334,"kb_avail":565919080,"statfs":{"total":579820584960,"available":579501137920,"internally_reserved":0,"allocated":14917632,"data_stored":6899106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":304470694},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003732"},"pg_stats":[{"pgid":"3.1f","version":"191'545","reported_seq":4874,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.220212+0000","last_change":"2024-09-19T21:38:20.220410+0000","last_active":"2024-09-19T21:38:20.220212+0000","last_peered":"2024-09-19T21:38:20.220212+0000","last_clean":"2024-09-19T21:38:20.220212+0000","last_became_active":"2024-09-19T21:37:09.888741+0000","last_became_peered":"2024-09-19T21:37:09.888741+0000","last_unstale":"2024-09-19T21:38:20.220212+0000","last_undegraded":"2024-09-19T21:38:20.220212+0000","last_fullsized":"2024-09-19T21:38:20.220212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'545","last_scrub_stamp":"2024-09-19T21:38:20.220176+0000","last_deep_scrub":"191'545","last_deep_scrub_stamp":"2024-09-19T21:38:20.220176+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.220176+0000","objects_scrubbed":6,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:30:48.946650+0000","scrub_duration":0.016243944,"objects_trimmed":0,"snaptrim_duration":0.0033110589999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6614,"num_read_kb":38674,"num_write":463,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":46520510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"191'295","reported_seq":2718,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.474184+0000","last_change":"2024-09-19T21:38:28.474184+0000","last_active":"2024-09-19T21:38:28.474184+0000","last_peered":"2024-09-19T21:38:28.474184+0000","last_clean":"2024-09-19T21:38:28.474184+0000","last_became_active":"2024-09-19T21:36:36.059076+0000","last_became_peered":"2024-09-19T21:36:36.059076+0000","last_unstale":"2024-09-19T21:38:28.474184+0000","last_undegraded":"2024-09-19T21:38:28.474184+0000","last_fullsized":"2024-09-19T21:38:28.474184+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'295","last_scrub_stamp":"2024-09-19T21:38:28.473931+0000","last_deep_scrub":"191'295","last_deep_scrub_stamp":"2024-09-19T21:38:28.473931+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.473931+0000","objects_scrubbed":14,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:50:21.235837+0000","scrub_duration":0.020161294999999999,"objects_trimmed":0,"snaptrim_duration":0.002116752,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1784,"num_read_kb":7799,"num_write":202,"num_write_kb":3445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3292808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"191'1571","reported_seq":10540,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:35.466271+0000","last_change":"2024-09-19T21:38:35.466385+0000","last_active":"2024-09-19T21:38:35.466271+0000","last_peered":"2024-09-19T21:38:35.466271+0000","last_clean":"2024-09-19T21:38:35.466271+0000","last_became_active":"2024-09-19T21:36:38.059347+0000","last_became_peered":"2024-09-19T21:36:38.059347+0000","last_unstale":"2024-09-19T21:38:35.466271+0000","last_undegraded":"2024-09-19T21:38:35.466271+0000","last_fullsized":"2024-09-19T21:38:35.466271+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1571","last_scrub_stamp":"2024-09-19T21:38:35.466247+0000","last_deep_scrub":"191'1571","last_deep_scrub_stamp":"2024-09-19T21:38:35.466247+0000","last_clean_scrub_stamp":"2024-09-19T21:38:35.466247+0000","objects_scrubbed":5,"log_size":1571,"log_dups_size":0,"ondisk_log_size":1571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:01:09.092433+0000","scrub_duration":0.016050299000000001,"objects_trimmed":0,"snaptrim_duration":0.0045447689999999997,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":17309,"num_read_kb":39375,"num_write":1977,"num_write_kb":15215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":23119424,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"191'669","reported_seq":16728,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:34.675555+0000","last_change":"2024-09-19T21:38:34.675667+0000","last_active":"2024-09-19T21:38:34.675555+0000","last_peered":"2024-09-19T21:38:34.675555+0000","last_clean":"2024-09-19T21:38:34.675555+0000","last_became_active":"2024-09-19T21:34:30.007272+0000","last_became_peered":"2024-09-19T21:34:30.007272+0000","last_unstale":"2024-09-19T21:38:34.675555+0000","last_undegraded":"2024-09-19T21:38:34.675555+0000","last_fullsized":"2024-09-19T21:38:34.675555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'669","last_scrub_stamp":"2024-09-19T21:38:34.675533+0000","last_deep_scrub":"191'669","last_deep_scrub_stamp":"2024-09-19T21:38:34.675533+0000","last_clean_scrub_stamp":"2024-09-19T21:38:34.675533+0000","objects_scrubbed":7,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:46:44.550188+0000","scrub_duration":0.016108365999999999,"objects_trimmed":0,"snaptrim_duration":0.003364672,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":17765,"num_read_kb":26868,"num_write":662,"num_write_kb":4155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"191'430","reported_seq":4629,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.436111+0000","last_change":"2024-09-19T21:38:29.436462+0000","last_active":"2024-09-19T21:38:29.436111+0000","last_peered":"2024-09-19T21:38:29.436111+0000","last_clean":"2024-09-19T21:38:29.436111+0000","last_became_active":"2024-09-19T21:34:30.003941+0000","last_became_peered":"2024-09-19T21:34:30.003941+0000","last_unstale":"2024-09-19T21:38:29.436111+0000","last_undegraded":"2024-09-19T21:38:29.436111+0000","last_fullsized":"2024-09-19T21:38:29.436111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'430","last_scrub_stamp":"2024-09-19T21:38:29.436062+0000","last_deep_scrub":"191'430","last_deep_scrub_stamp":"2024-09-19T21:38:29.436062+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.436062+0000","objects_scrubbed":9,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:11:13.461944+0000","scrub_duration":0.016310802999999999,"objects_trimmed":0,"snaptrim_duration":0.001429388,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4146,"num_read_kb":25859,"num_write":373,"num_write_kb":6029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"191'1025","reported_seq":9323,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:32.641312+0000","last_change":"2024-09-19T21:38:32.641436+0000","last_active":"2024-09-19T21:38:32.641312+0000","last_peered":"2024-09-19T21:38:32.641312+0000","last_clean":"2024-09-19T21:38:32.641312+0000","last_became_active":"2024-09-19T21:37:12.843721+0000","last_became_peered":"2024-09-19T21:37:12.843721+0000","last_unstale":"2024-09-19T21:38:32.641312+0000","last_undegraded":"2024-09-19T21:38:32.641312+0000","last_fullsized":"2024-09-19T21:38:32.641312+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1025","last_scrub_stamp":"2024-09-19T21:38:32.641289+0000","last_deep_scrub":"191'1025","last_deep_scrub_stamp":"2024-09-19T21:38:32.641289+0000","last_clean_scrub_stamp":"2024-09-19T21:38:32.641289+0000","objects_scrubbed":9,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:42:11.228503+0000","scrub_duration":0.022733077000000001,"objects_trimmed":0,"snaptrim_duration":0.0029868080000000001,"stat_sum":{"num_bytes":19,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":11962,"num_read_kb":38247,"num_write":1471,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":9376052,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"191'442","reported_seq":14890,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:22.110968+0000","last_change":"2024-09-19T21:38:22.111093+0000","last_active":"2024-09-19T21:38:22.110968+0000","last_peered":"2024-09-19T21:38:22.110968+0000","last_clean":"2024-09-19T21:38:22.110968+0000","last_became_active":"2024-09-19T21:37:52.856015+0000","last_became_peered":"2024-09-19T21:37:52.856015+0000","last_unstale":"2024-09-19T21:38:22.110968+0000","last_undegraded":"2024-09-19T21:38:22.110968+0000","last_fullsized":"2024-09-19T21:38:22.110968+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'442","last_scrub_stamp":"2024-09-19T21:38:22.110946+0000","last_deep_scrub":"191'442","last_deep_scrub_stamp":"2024-09-19T21:38:22.110946+0000","last_clean_scrub_stamp":"2024-09-19T21:38:22.110946+0000","objects_scrubbed":7,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:47:57.884771+0000","scrub_duration":0.012774656000000001,"objects_trimmed":0,"snaptrim_duration":0.001253461,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":14847,"num_read_kb":34110,"num_write":485,"num_write_kb":8777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":176,"num_bytes_recovered":7196866,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"191'351","reported_seq":10962,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:25.488308+0000","last_change":"2024-09-19T21:38:25.488455+0000","last_active":"2024-09-19T21:38:25.488308+0000","last_peered":"2024-09-19T21:38:25.488308+0000","last_clean":"2024-09-19T21:38:25.488308+0000","last_became_active":"2024-09-19T21:37:51.870223+0000","last_became_peered":"2024-09-19T21:37:51.870223+0000","last_unstale":"2024-09-19T21:38:25.488308+0000","last_undegraded":"2024-09-19T21:38:25.488308+0000","last_fullsized":"2024-09-19T21:38:25.488308+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'351","last_scrub_stamp":"2024-09-19T21:38:25.488287+0000","last_deep_scrub":"191'351","last_deep_scrub_stamp":"2024-09-19T21:38:25.488287+0000","last_clean_scrub_stamp":"2024-09-19T21:38:25.488287+0000","objects_scrubbed":10,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:17:54.432575+0000","scrub_duration":0.01809614,"objects_trimmed":0,"snaptrim_duration":0.001011975,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10865,"num_read_kb":42398,"num_write":381,"num_write_kb":14208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":3413566,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"191'338","reported_seq":6465,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.365960+0000","last_change":"2024-09-19T21:38:26.366104+0000","last_active":"2024-09-19T21:38:26.365960+0000","last_peered":"2024-09-19T21:38:26.365960+0000","last_clean":"2024-09-19T21:38:26.365960+0000","last_became_active":"2024-09-19T21:37:10.890617+0000","last_became_peered":"2024-09-19T21:37:10.890617+0000","last_unstale":"2024-09-19T21:38:26.365960+0000","last_undegraded":"2024-09-19T21:38:26.365960+0000","last_fullsized":"2024-09-19T21:38:26.365960+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'338","last_scrub_stamp":"2024-09-19T21:38:26.365933+0000","last_deep_scrub":"191'338","last_deep_scrub_stamp":"2024-09-19T21:38:26.365933+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.365933+0000","objects_scrubbed":10,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:05.018045+0000","scrub_duration":0.017908897999999999,"objects_trimmed":0,"snaptrim_duration":0.0010987029999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6201,"num_read_kb":14803,"num_write":229,"num_write_kb":4540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":4145152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"191'391","reported_seq":11311,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:34.425443+0000","last_change":"2024-09-19T21:38:34.425574+0000","last_active":"2024-09-19T21:38:34.425443+0000","last_peered":"2024-09-19T21:38:34.425443+0000","last_clean":"2024-09-19T21:38:34.425443+0000","last_became_active":"2024-09-19T21:36:35.055148+0000","last_became_peered":"2024-09-19T21:36:35.055148+0000","last_unstale":"2024-09-19T21:38:34.425443+0000","last_undegraded":"2024-09-19T21:38:34.425443+0000","last_fullsized":"2024-09-19T21:38:34.425443+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'391","last_scrub_stamp":"2024-09-19T21:38:34.425419+0000","last_deep_scrub":"191'391","last_deep_scrub_stamp":"2024-09-19T21:38:34.425419+0000","last_clean_scrub_stamp":"2024-09-19T21:38:34.425419+0000","objects_scrubbed":12,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:22:54.375552+0000","scrub_duration":0.020172529000000002,"objects_trimmed":0,"snaptrim_duration":0.0049046749999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":12241,"num_read_kb":30455,"num_write":288,"num_write_kb":6850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":9070672,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"191'420","reported_seq":7058,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.284875+0000","last_change":"2024-09-19T21:38:21.285051+0000","last_active":"2024-09-19T21:38:21.284875+0000","last_peered":"2024-09-19T21:38:21.284875+0000","last_clean":"2024-09-19T21:38:21.284875+0000","last_became_active":"2024-09-19T21:36:05.845764+0000","last_became_peered":"2024-09-19T21:36:05.845764+0000","last_unstale":"2024-09-19T21:38:21.284875+0000","last_undegraded":"2024-09-19T21:38:21.284875+0000","last_fullsized":"2024-09-19T21:38:21.284875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'420","last_scrub_stamp":"2024-09-19T21:38:21.284849+0000","last_deep_scrub":"191'420","last_deep_scrub_stamp":"2024-09-19T21:38:21.284849+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.284849+0000","objects_scrubbed":11,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:14:33.158842+0000","scrub_duration":0.022286953000000002,"objects_trimmed":0,"snaptrim_duration":0.001244797,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6712,"num_read_kb":12124,"num_write":299,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":785390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"191'381","reported_seq":10423,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:39.671616+0000","last_change":"2024-09-19T21:38:39.671783+0000","last_active":"2024-09-19T21:38:39.671616+0000","last_peered":"2024-09-19T21:38:39.671616+0000","last_clean":"2024-09-19T21:38:39.671616+0000","last_became_active":"2024-09-19T21:37:59.932435+0000","last_became_peered":"2024-09-19T21:37:59.932435+0000","last_unstale":"2024-09-19T21:38:39.671616+0000","last_undegraded":"2024-09-19T21:38:39.671616+0000","last_fullsized":"2024-09-19T21:38:39.671616+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'381","last_scrub_stamp":"2024-09-19T21:38:39.671565+0000","last_deep_scrub":"191'381","last_deep_scrub_stamp":"2024-09-19T21:38:39.671565+0000","last_clean_scrub_stamp":"2024-09-19T21:38:39.671565+0000","objects_scrubbed":14,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:07:32.630285+0000","scrub_duration":0.021703704000000001,"objects_trimmed":0,"snaptrim_duration":0.0036488620000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":9810,"num_read_kb":27295,"num_write":283,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":9108528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:36.709231+0000","last_change":"2024-09-19T21:38:36.709471+0000","last_active":"2024-09-19T21:38:36.709231+0000","last_peered":"2024-09-19T21:38:36.709231+0000","last_clean":"2024-09-19T21:38:36.709231+0000","last_became_active":"2024-09-19T21:37:56.901828+0000","last_became_peered":"2024-09-19T21:37:56.901828+0000","last_unstale":"2024-09-19T21:38:36.709231+0000","last_undegraded":"2024-09-19T21:38:36.709231+0000","last_fullsized":"2024-09-19T21:38:36.709231+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:36.709182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:36.709182+0000","last_clean_scrub_stamp":"2024-09-19T21:38:36.709182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:16.900010+0000","scrub_duration":0.0032374019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"191'602","reported_seq":9051,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.168524+0000","last_change":"2024-09-19T21:38:29.168706+0000","last_active":"2024-09-19T21:38:29.168524+0000","last_peered":"2024-09-19T21:38:29.168524+0000","last_clean":"2024-09-19T21:38:29.168524+0000","last_became_active":"2024-09-19T21:37:51.869560+0000","last_became_peered":"2024-09-19T21:37:51.869560+0000","last_unstale":"2024-09-19T21:38:29.168524+0000","last_undegraded":"2024-09-19T21:38:29.168524+0000","last_fullsized":"2024-09-19T21:38:29.168524+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'602","last_scrub_stamp":"2024-09-19T21:38:29.168490+0000","last_deep_scrub":"191'602","last_deep_scrub_stamp":"2024-09-19T21:38:29.168490+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.168490+0000","objects_scrubbed":4,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:29:38.267584+0000","scrub_duration":0.012096908999999999,"objects_trimmed":10,"snaptrim_duration":0.002026169,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":8954,"num_read_kb":23692,"num_write":472,"num_write_kb":6488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":2964557,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.150002+0000","last_change":"2024-09-19T21:38:21.150321+0000","last_active":"2024-09-19T21:38:21.150002+0000","last_peered":"2024-09-19T21:38:21.150002+0000","last_clean":"2024-09-19T21:38:21.150002+0000","last_became_active":"2024-09-19T21:37:48.856078+0000","last_became_peered":"2024-09-19T21:37:48.856078+0000","last_unstale":"2024-09-19T21:38:21.150002+0000","last_undegraded":"2024-09-19T21:38:21.150002+0000","last_fullsized":"2024-09-19T21:38:21.150002+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:21.149956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:21.149956+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.149956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:36:46.810807+0000","scrub_duration":0.0034548930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"191'781","reported_seq":9112,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:34.182596+0000","last_change":"2024-09-19T21:38:34.182730+0000","last_active":"2024-09-19T21:38:34.182596+0000","last_peered":"2024-09-19T21:38:34.182596+0000","last_clean":"2024-09-19T21:38:34.182596+0000","last_became_active":"2024-09-19T21:37:53.867514+0000","last_became_peered":"2024-09-19T21:37:53.867514+0000","last_unstale":"2024-09-19T21:38:34.182596+0000","last_undegraded":"2024-09-19T21:38:34.182596+0000","last_fullsized":"2024-09-19T21:38:34.182596+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'781","last_scrub_stamp":"2024-09-19T21:38:34.182572+0000","last_deep_scrub":"191'781","last_deep_scrub_stamp":"2024-09-19T21:38:34.182572+0000","last_clean_scrub_stamp":"2024-09-19T21:38:34.182572+0000","objects_scrubbed":9,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:59:25.361433+0000","scrub_duration":0.01690471,"objects_trimmed":0,"snaptrim_duration":0.00083092500000000004,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10761,"num_read_kb":42234,"num_write":834,"num_write_kb":10720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":2404929,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:21.595490+0000","last_change":"2024-09-19T21:38:21.595490+0000","last_active":"2024-09-19T21:38:21.595490+0000","last_peered":"2024-09-19T21:38:21.595490+0000","last_clean":"2024-09-19T21:38:21.595490+0000","last_became_active":"2024-09-19T21:37:48.860073+0000","last_became_peered":"2024-09-19T21:37:48.860073+0000","last_unstale":"2024-09-19T21:38:21.595490+0000","last_undegraded":"2024-09-19T21:38:21.595490+0000","last_fullsized":"2024-09-19T21:38:21.595490+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:21.595049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:21.595049+0000","last_clean_scrub_stamp":"2024-09-19T21:38:21.595049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:08:55.022948+0000","scrub_duration":0.003699894,"objects_trimmed":0,"snaptrim_duration":0.0010730690000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"191'543","reported_seq":13402,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:27.408581+0000","last_change":"2024-09-19T21:38:27.408818+0000","last_active":"2024-09-19T21:38:27.408581+0000","last_peered":"2024-09-19T21:38:27.408581+0000","last_clean":"2024-09-19T21:38:27.408581+0000","last_became_active":"2024-09-19T21:37:30.005886+0000","last_became_peered":"2024-09-19T21:37:30.005886+0000","last_unstale":"2024-09-19T21:38:27.408581+0000","last_undegraded":"2024-09-19T21:38:27.408581+0000","last_fullsized":"2024-09-19T21:38:27.408581+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'543","last_scrub_stamp":"2024-09-19T21:38:27.408516+0000","last_deep_scrub":"191'543","last_deep_scrub_stamp":"2024-09-19T21:38:27.408516+0000","last_clean_scrub_stamp":"2024-09-19T21:38:27.408516+0000","objects_scrubbed":10,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:18:42.745523+0000","scrub_duration":0.019168872999999999,"objects_trimmed":0,"snaptrim_duration":0.0011934580000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":12540,"num_read_kb":41393,"num_write":388,"num_write_kb":7022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":29241055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:29.429345+0000","last_change":"2024-09-19T21:38:29.429734+0000","last_active":"2024-09-19T21:38:29.429345+0000","last_peered":"2024-09-19T21:38:29.429345+0000","last_clean":"2024-09-19T21:38:29.429345+0000","last_became_active":"2024-09-19T21:38:06.001133+0000","last_became_peered":"2024-09-19T21:38:06.001133+0000","last_unstale":"2024-09-19T21:38:29.429345+0000","last_undegraded":"2024-09-19T21:38:29.429345+0000","last_fullsized":"2024-09-19T21:38:29.429345+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:29.429256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:29.429256+0000","last_clean_scrub_stamp":"2024-09-19T21:38:29.429256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:02:17.288111+0000","scrub_duration":0.0033198419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"191'893","reported_seq":5720,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:23.352176+0000","last_change":"2024-09-19T21:38:23.352176+0000","last_active":"2024-09-19T21:38:23.352176+0000","last_peered":"2024-09-19T21:38:23.352176+0000","last_clean":"2024-09-19T21:38:23.352176+0000","last_became_active":"2024-09-19T21:36:37.060314+0000","last_became_peered":"2024-09-19T21:36:37.060314+0000","last_unstale":"2024-09-19T21:38:23.352176+0000","last_undegraded":"2024-09-19T21:38:23.352176+0000","last_fullsized":"2024-09-19T21:38:23.352176+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'893","last_scrub_stamp":"2024-09-19T21:38:23.351971+0000","last_deep_scrub":"191'893","last_deep_scrub_stamp":"2024-09-19T21:38:23.351971+0000","last_clean_scrub_stamp":"2024-09-19T21:38:23.351971+0000","objects_scrubbed":11,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:07:24.468656+0000","scrub_duration":0.022058634000000001,"objects_trimmed":0,"snaptrim_duration":0.0034173129999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":8516,"num_read_kb":22658,"num_write":1079,"num_write_kb":8304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":7596472,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"191'1038","reported_seq":10122,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:34.102013+0000","last_change":"2024-09-19T21:38:34.102013+0000","last_active":"2024-09-19T21:38:34.102013+0000","last_peered":"2024-09-19T21:38:34.102013+0000","last_clean":"2024-09-19T21:38:34.102013+0000","last_became_active":"2024-09-19T21:36:37.061649+0000","last_became_peered":"2024-09-19T21:36:37.061649+0000","last_unstale":"2024-09-19T21:38:34.102013+0000","last_undegraded":"2024-09-19T21:38:34.102013+0000","last_fullsized":"2024-09-19T21:38:34.102013+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'1038","last_scrub_stamp":"2024-09-19T21:38:34.101760+0000","last_deep_scrub":"191'1038","last_deep_scrub_stamp":"2024-09-19T21:38:34.101760+0000","last_clean_scrub_stamp":"2024-09-19T21:38:34.101760+0000","objects_scrubbed":9,"log_size":1038,"log_dups_size":0,"ondisk_log_size":1038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:28.797092+0000","scrub_duration":0.017501188000000001,"objects_trimmed":0,"snaptrim_duration":0.001983917,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":13971,"num_read_kb":21772,"num_write":1059,"num_write_kb":5006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":4539466,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"191'3","reported_seq":249,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:18.243818+0000","last_change":"2024-09-19T21:38:18.243818+0000","last_active":"2024-09-19T21:38:18.243818+0000","last_peered":"2024-09-19T21:38:18.243818+0000","last_clean":"2024-09-19T21:38:18.243818+0000","last_became_active":"2024-09-19T21:37:48.857689+0000","last_became_peered":"2024-09-19T21:37:48.857689+0000","last_unstale":"2024-09-19T21:38:18.243818+0000","last_undegraded":"2024-09-19T21:38:18.243818+0000","last_fullsized":"2024-09-19T21:38:18.243818+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'3","last_scrub_stamp":"2024-09-19T21:38:18.243601+0000","last_deep_scrub":"191'3","last_deep_scrub_stamp":"2024-09-19T21:38:18.243601+0000","last_clean_scrub_stamp":"2024-09-19T21:38:18.243601+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:52:47.473469+0000","scrub_duration":0.0064841209999999998,"objects_trimmed":0,"snaptrim_duration":0.00092979599999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"191'581","reported_seq":12116,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:22.241787+0000","last_change":"2024-09-19T21:38:22.241925+0000","last_active":"2024-09-19T21:38:22.241787+0000","last_peered":"2024-09-19T21:38:22.241787+0000","last_clean":"2024-09-19T21:38:22.241787+0000","last_became_active":"2024-09-19T21:37:10.888976+0000","last_became_peered":"2024-09-19T21:37:10.888976+0000","last_unstale":"2024-09-19T21:38:22.241787+0000","last_undegraded":"2024-09-19T21:38:22.241787+0000","last_fullsized":"2024-09-19T21:38:22.241787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'581","last_scrub_stamp":"2024-09-19T21:38:22.241766+0000","last_deep_scrub":"191'581","last_deep_scrub_stamp":"2024-09-19T21:38:22.241766+0000","last_clean_scrub_stamp":"2024-09-19T21:38:22.241766+0000","objects_scrubbed":11,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:02:43.033390+0000","scrub_duration":0.018496238000000002,"objects_trimmed":0,"snaptrim_duration":0.002401065,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13287,"num_read_kb":50555,"num_write":517,"num_write_kb":12214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":21449636,"num_keys_recovered":118,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.474067+0000","last_change":"2024-09-19T21:38:20.474067+0000","last_active":"2024-09-19T21:38:20.474067+0000","last_peered":"2024-09-19T21:38:20.474067+0000","last_clean":"2024-09-19T21:38:20.474067+0000","last_became_active":"2024-09-19T21:37:06.857180+0000","last_became_peered":"2024-09-19T21:37:06.857180+0000","last_unstale":"2024-09-19T21:38:20.474067+0000","last_undegraded":"2024-09-19T21:38:20.474067+0000","last_fullsized":"2024-09-19T21:38:20.474067+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:20.473691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:20.473691+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.473691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:48:31.793308+0000","scrub_duration":0.0028180929999999998,"objects_trimmed":0,"snaptrim_duration":0.00068062899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"191'781","reported_seq":9259,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.183841+0000","last_change":"2024-09-19T21:38:28.184044+0000","last_active":"2024-09-19T21:38:28.183841+0000","last_peered":"2024-09-19T21:38:28.183841+0000","last_clean":"2024-09-19T21:38:28.183841+0000","last_became_active":"2024-09-19T21:37:52.860009+0000","last_became_peered":"2024-09-19T21:37:52.860009+0000","last_unstale":"2024-09-19T21:38:28.183841+0000","last_undegraded":"2024-09-19T21:38:28.183841+0000","last_fullsized":"2024-09-19T21:38:28.183841+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'781","last_scrub_stamp":"2024-09-19T21:38:28.183785+0000","last_deep_scrub":"191'781","last_deep_scrub_stamp":"2024-09-19T21:38:28.183785+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.183785+0000","objects_scrubbed":9,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:27:10.956335+0000","scrub_duration":0.018140495,"objects_trimmed":0,"snaptrim_duration":0.007010049,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9933,"num_read_kb":35483,"num_write":987,"num_write_kb":10420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":19189871,"num_keys_recovered":27,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.476160+0000","last_change":"2024-09-19T21:38:30.476160+0000","last_active":"2024-09-19T21:38:30.476160+0000","last_peered":"2024-09-19T21:38:30.476160+0000","last_clean":"2024-09-19T21:38:30.476160+0000","last_became_active":"2024-09-19T21:37:06.860810+0000","last_became_peered":"2024-09-19T21:37:06.860810+0000","last_unstale":"2024-09-19T21:38:30.476160+0000","last_undegraded":"2024-09-19T21:38:30.476160+0000","last_fullsized":"2024-09-19T21:38:30.476160+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T21:38:30.475855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T21:38:30.475855+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.475855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:56:03.185387+0000","scrub_duration":0.0032581400000000001,"objects_trimmed":0,"snaptrim_duration":0.00041290800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"191'503","reported_seq":12921,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.387987+0000","last_change":"2024-09-19T21:38:30.387987+0000","last_active":"2024-09-19T21:38:30.387987+0000","last_peered":"2024-09-19T21:38:30.387987+0000","last_clean":"2024-09-19T21:38:30.387987+0000","last_became_active":"2024-09-19T21:36:35.049591+0000","last_became_peered":"2024-09-19T21:36:35.049591+0000","last_unstale":"2024-09-19T21:38:30.387987+0000","last_undegraded":"2024-09-19T21:38:30.387987+0000","last_fullsized":"2024-09-19T21:38:30.387987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'503","last_scrub_stamp":"2024-09-19T21:38:30.387492+0000","last_deep_scrub":"191'503","last_deep_scrub_stamp":"2024-09-19T21:38:30.387492+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.387492+0000","objects_scrubbed":7,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:17.311069+0000","scrub_duration":0.015325646,"objects_trimmed":0,"snaptrim_duration":0.001053691,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13696,"num_read_kb":48732,"num_write":499,"num_write_kb":3509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":2666584,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"191'2","reported_seq":226,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:25.579128+0000","last_change":"2024-09-19T21:38:25.579128+0000","last_active":"2024-09-19T21:38:25.579128+0000","last_peered":"2024-09-19T21:38:25.579128+0000","last_clean":"2024-09-19T21:38:25.579128+0000","last_became_active":"2024-09-19T21:36:32.034600+0000","last_became_peered":"2024-09-19T21:36:32.034600+0000","last_unstale":"2024-09-19T21:38:25.579128+0000","last_undegraded":"2024-09-19T21:38:25.579128+0000","last_fullsized":"2024-09-19T21:38:25.579128+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'2","last_scrub_stamp":"2024-09-19T21:38:25.578844+0000","last_deep_scrub":"191'2","last_deep_scrub_stamp":"2024-09-19T21:38:25.578844+0000","last_clean_scrub_stamp":"2024-09-19T21:38:25.578844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T09:12:57.236017+0000","scrub_duration":0.0061060630000000001,"objects_trimmed":0,"snaptrim_duration":0.001355396,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"191'119","reported_seq":434,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.559461+0000","last_change":"2024-09-19T21:38:20.559823+0000","last_active":"2024-09-19T21:38:20.559461+0000","last_peered":"2024-09-19T21:38:20.559461+0000","last_clean":"2024-09-19T21:38:20.559461+0000","last_became_active":"2024-09-19T21:37:11.904073+0000","last_became_peered":"2024-09-19T21:37:11.904073+0000","last_unstale":"2024-09-19T21:38:20.559461+0000","last_undegraded":"2024-09-19T21:38:20.559461+0000","last_fullsized":"2024-09-19T21:38:20.559461+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'119","last_scrub_stamp":"2024-09-19T21:38:20.559345+0000","last_deep_scrub":"191'119","last_deep_scrub_stamp":"2024-09-19T21:38:20.559345+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.559345+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:00:37.680058+0000","scrub_duration":0.011967459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"191'559","reported_seq":15987,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:18.122470+0000","last_change":"2024-09-19T21:38:18.122594+0000","last_active":"2024-09-19T21:38:18.122470+0000","last_peered":"2024-09-19T21:38:18.122470+0000","last_clean":"2024-09-19T21:38:18.122470+0000","last_became_active":"2024-09-19T21:37:51.869925+0000","last_became_peered":"2024-09-19T21:37:51.869925+0000","last_unstale":"2024-09-19T21:38:18.122470+0000","last_undegraded":"2024-09-19T21:38:18.122470+0000","last_fullsized":"2024-09-19T21:38:18.122470+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'559","last_scrub_stamp":"2024-09-19T21:38:18.122448+0000","last_deep_scrub":"191'559","last_deep_scrub_stamp":"2024-09-19T21:38:18.122448+0000","last_clean_scrub_stamp":"2024-09-19T21:38:18.122448+0000","objects_scrubbed":12,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:49:13.665052+0000","scrub_duration":0.018890111000000001,"objects_trimmed":0,"snaptrim_duration":0.0052492449999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14449,"num_read_kb":62425,"num_write":569,"num_write_kb":13072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":25134328,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"191'464","reported_seq":5856,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:41.229105+0000","last_change":"2024-09-19T21:38:41.229105+0000","last_active":"2024-09-19T21:38:41.229105+0000","last_peered":"2024-09-19T21:38:41.229105+0000","last_clean":"2024-09-19T21:38:41.229105+0000","last_became_active":"2024-09-19T21:34:29.998402+0000","last_became_peered":"2024-09-19T21:34:29.998402+0000","last_unstale":"2024-09-19T21:38:41.229105+0000","last_undegraded":"2024-09-19T21:38:41.229105+0000","last_fullsized":"2024-09-19T21:38:41.229105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'464","last_scrub_stamp":"2024-09-19T21:38:41.228918+0000","last_deep_scrub":"191'464","last_deep_scrub_stamp":"2024-09-19T21:38:41.228918+0000","last_clean_scrub_stamp":"2024-09-19T21:38:41.228918+0000","objects_scrubbed":7,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T08:09:42.201488+0000","scrub_duration":0.018866919999999999,"objects_trimmed":0,"snaptrim_duration":0.00096270600000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6022,"num_read_kb":18872,"num_write":390,"num_write_kb":6852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"191'682","reported_seq":9192,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:19.226614+0000","last_change":"2024-09-19T21:38:19.226614+0000","last_active":"2024-09-19T21:38:19.226614+0000","last_peered":"2024-09-19T21:38:19.226614+0000","last_clean":"2024-09-19T21:38:19.226614+0000","last_became_active":"2024-09-19T21:37:59.931820+0000","last_became_peered":"2024-09-19T21:37:59.931820+0000","last_unstale":"2024-09-19T21:38:19.226614+0000","last_undegraded":"2024-09-19T21:38:19.226614+0000","last_fullsized":"2024-09-19T21:38:19.226614+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'682","last_scrub_stamp":"2024-09-19T21:38:19.226361+0000","last_deep_scrub":"191'682","last_deep_scrub_stamp":"2024-09-19T21:38:19.226361+0000","last_clean_scrub_stamp":"2024-09-19T21:38:19.226361+0000","objects_scrubbed":15,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:43:36.026665+0000","scrub_duration":0.020747162999999999,"objects_trimmed":0,"snaptrim_duration":0.00087677199999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":10041,"num_read_kb":47894,"num_write":722,"num_write_kb":13342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":12010123,"num_keys_recovered":38,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"191'335","reported_seq":2635,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.460435+0000","last_change":"2024-09-19T21:38:26.460755+0000","last_active":"2024-09-19T21:38:26.460435+0000","last_peered":"2024-09-19T21:38:26.460435+0000","last_clean":"2024-09-19T21:38:26.460435+0000","last_became_active":"2024-09-19T21:34:30.004137+0000","last_became_peered":"2024-09-19T21:34:30.004137+0000","last_unstale":"2024-09-19T21:38:26.460435+0000","last_undegraded":"2024-09-19T21:38:26.460435+0000","last_fullsized":"2024-09-19T21:38:26.460435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'335","last_scrub_stamp":"2024-09-19T21:38:26.460374+0000","last_deep_scrub":"191'335","last_deep_scrub_stamp":"2024-09-19T21:38:26.460374+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.460374+0000","objects_scrubbed":10,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T07:29:19.610517+0000","scrub_duration":0.015993165,"objects_trimmed":0,"snaptrim_duration":0.001728063,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2331,"num_read_kb":21251,"num_write":328,"num_write_kb":7577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"191'324","reported_seq":4069,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:35.098418+0000","last_change":"2024-09-19T21:38:35.098566+0000","last_active":"2024-09-19T21:38:35.098418+0000","last_peered":"2024-09-19T21:38:35.098418+0000","last_clean":"2024-09-19T21:38:35.098418+0000","last_became_active":"2024-09-19T21:34:30.008987+0000","last_became_peered":"2024-09-19T21:34:30.008987+0000","last_unstale":"2024-09-19T21:38:35.098418+0000","last_undegraded":"2024-09-19T21:38:35.098418+0000","last_fullsized":"2024-09-19T21:38:35.098418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'324","last_scrub_stamp":"2024-09-19T21:38:35.098392+0000","last_deep_scrub":"191'324","last_deep_scrub_stamp":"2024-09-19T21:38:35.098392+0000","last_clean_scrub_stamp":"2024-09-19T21:38:35.098392+0000","objects_scrubbed":6,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T04:32:11.118417+0000","scrub_duration":0.013277860000000001,"objects_trimmed":0,"snaptrim_duration":0.0013234640000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3740,"num_read_kb":13556,"num_write":250,"num_write_kb":4462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"191'511","reported_seq":9729,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:30.645640+0000","last_change":"2024-09-19T21:38:30.645836+0000","last_active":"2024-09-19T21:38:30.645640+0000","last_peered":"2024-09-19T21:38:30.645640+0000","last_clean":"2024-09-19T21:38:30.645640+0000","last_became_active":"2024-09-19T21:34:30.007560+0000","last_became_peered":"2024-09-19T21:34:30.007560+0000","last_unstale":"2024-09-19T21:38:30.645640+0000","last_undegraded":"2024-09-19T21:38:30.645640+0000","last_fullsized":"2024-09-19T21:38:30.645640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'511","last_scrub_stamp":"2024-09-19T21:38:30.645613+0000","last_deep_scrub":"191'511","last_deep_scrub_stamp":"2024-09-19T21:38:30.645613+0000","last_clean_scrub_stamp":"2024-09-19T21:38:30.645613+0000","objects_scrubbed":11,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T00:38:28.104714+0000","scrub_duration":0.020534383,"objects_trimmed":0,"snaptrim_duration":0.002880259,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9431,"num_read_kb":32116,"num_write":486,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"191'598","reported_seq":6455,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:24.357021+0000","last_change":"2024-09-19T21:38:24.357021+0000","last_active":"2024-09-19T21:38:24.357021+0000","last_peered":"2024-09-19T21:38:24.357021+0000","last_clean":"2024-09-19T21:38:24.357021+0000","last_became_active":"2024-09-19T21:36:36.058292+0000","last_became_peered":"2024-09-19T21:36:36.058292+0000","last_unstale":"2024-09-19T21:38:24.357021+0000","last_undegraded":"2024-09-19T21:38:24.357021+0000","last_fullsized":"2024-09-19T21:38:24.357021+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'598","last_scrub_stamp":"2024-09-19T21:38:24.356738+0000","last_deep_scrub":"191'598","last_deep_scrub_stamp":"2024-09-19T21:38:24.356738+0000","last_clean_scrub_stamp":"2024-09-19T21:38:24.356738+0000","objects_scrubbed":10,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T05:23:30.351676+0000","scrub_duration":0.018075013000000001,"objects_trimmed":0,"snaptrim_duration":0.0053704809999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8318,"num_read_kb":51766,"num_write":483,"num_write_kb":7407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":2854504,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"191'870","reported_seq":5616,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:20.251980+0000","last_change":"2024-09-19T21:38:20.252100+0000","last_active":"2024-09-19T21:38:20.251980+0000","last_peered":"2024-09-19T21:38:20.251980+0000","last_clean":"2024-09-19T21:38:20.251980+0000","last_became_active":"2024-09-19T21:37:59.932188+0000","last_became_peered":"2024-09-19T21:37:59.932188+0000","last_unstale":"2024-09-19T21:38:20.251980+0000","last_undegraded":"2024-09-19T21:38:20.251980+0000","last_fullsized":"2024-09-19T21:38:20.251980+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'870","last_scrub_stamp":"2024-09-19T21:38:20.251961+0000","last_deep_scrub":"191'870","last_deep_scrub_stamp":"2024-09-19T21:38:20.251961+0000","last_clean_scrub_stamp":"2024-09-19T21:38:20.251961+0000","objects_scrubbed":12,"log_size":870,"log_dups_size":0,"ondisk_log_size":870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:15:33.610802+0000","scrub_duration":0.020965451,"objects_trimmed":0,"snaptrim_duration":0.001434488,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7566,"num_read_kb":14512,"num_write":906,"num_write_kb":4950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":892583,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"191'627","reported_seq":8825,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:26.645022+0000","last_change":"2024-09-19T21:38:26.645246+0000","last_active":"2024-09-19T21:38:26.645022+0000","last_peered":"2024-09-19T21:38:26.645022+0000","last_clean":"2024-09-19T21:38:26.645022+0000","last_became_active":"2024-09-19T21:37:11.903992+0000","last_became_peered":"2024-09-19T21:37:11.903992+0000","last_unstale":"2024-09-19T21:38:26.645022+0000","last_undegraded":"2024-09-19T21:38:26.645022+0000","last_fullsized":"2024-09-19T21:38:26.645022+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'627","last_scrub_stamp":"2024-09-19T21:38:26.644980+0000","last_deep_scrub":"191'627","last_deep_scrub_stamp":"2024-09-19T21:38:26.644980+0000","last_clean_scrub_stamp":"2024-09-19T21:38:26.644980+0000","objects_scrubbed":21,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T06:21:20.941788+0000","scrub_duration":0.026630226999999999,"objects_trimmed":0,"snaptrim_duration":0.0031036919999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10167,"num_read_kb":59570,"num_write":520,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":27638926,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"191'445","reported_seq":4273,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:17.435419+0000","last_change":"2024-09-19T21:38:17.435419+0000","last_active":"2024-09-19T21:38:17.435419+0000","last_peered":"2024-09-19T21:38:17.435419+0000","last_clean":"2024-09-19T21:38:17.435419+0000","last_became_active":"2024-09-19T21:36:35.055495+0000","last_became_peered":"2024-09-19T21:36:35.055495+0000","last_unstale":"2024-09-19T21:38:17.435419+0000","last_undegraded":"2024-09-19T21:38:17.435419+0000","last_fullsized":"2024-09-19T21:38:17.435419+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'445","last_scrub_stamp":"2024-09-19T21:38:17.435226+0000","last_deep_scrub":"191'445","last_deep_scrub_stamp":"2024-09-19T21:38:17.435226+0000","last_clean_scrub_stamp":"2024-09-19T21:38:17.435226+0000","objects_scrubbed":10,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T03:11:54.744906+0000","scrub_duration":0.016458093,"objects_trimmed":0,"snaptrim_duration":0.0056633609999999996,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4613,"num_read_kb":20668,"num_write":305,"num_write_kb":2921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":2643528,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"191'309","reported_seq":7510,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:19.182295+0000","last_change":"2024-09-19T21:38:19.182444+0000","last_active":"2024-09-19T21:38:19.182295+0000","last_peered":"2024-09-19T21:38:19.182295+0000","last_clean":"2024-09-19T21:38:19.182295+0000","last_became_active":"2024-09-19T21:37:11.894346+0000","last_became_peered":"2024-09-19T21:37:11.894346+0000","last_unstale":"2024-09-19T21:38:19.182295+0000","last_undegraded":"2024-09-19T21:38:19.182295+0000","last_fullsized":"2024-09-19T21:38:19.182295+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'309","last_scrub_stamp":"2024-09-19T21:38:19.182268+0000","last_deep_scrub":"191'309","last_deep_scrub_stamp":"2024-09-19T21:38:19.182268+0000","last_clean_scrub_stamp":"2024-09-19T21:38:19.182268+0000","objects_scrubbed":12,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T02:22:24.858075+0000","scrub_duration":0.019684788000000002,"objects_trimmed":0,"snaptrim_duration":0.0023440230000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6850,"num_read_kb":72074,"num_write":291,"num_write_kb":8598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":5123072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"191'431","reported_seq":7938,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-19T21:38:28.689136+0000","last_change":"2024-09-19T21:38:28.689414+0000","last_active":"2024-09-19T21:38:28.689136+0000","last_peered":"2024-09-19T21:38:28.689136+0000","last_clean":"2024-09-19T21:38:28.689136+0000","last_became_active":"2024-09-19T21:34:30.006729+0000","last_became_peered":"2024-09-19T21:34:30.006729+0000","last_unstale":"2024-09-19T21:38:28.689136+0000","last_undegraded":"2024-09-19T21:38:28.689136+0000","last_fullsized":"2024-09-19T21:38:28.689136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"191'431","last_scrub_stamp":"2024-09-19T21:38:28.689090+0000","last_deep_scrub":"191'431","last_deep_scrub_stamp":"2024-09-19T21:38:28.689090+0000","last_clean_scrub_stamp":"2024-09-19T21:38:28.689090+0000","objects_scrubbed":15,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T01:46:34.172771+0000","scrub_duration":0.024207152999999999,"objects_trimmed":0,"snaptrim_duration":0.0024867299999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8193,"num_read_kb":20785,"num_write":406,"num_write_kb":5738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":325,"num_object_clones":0,"num_object_copies":650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":325,"num_whiteouts":321,"num_read":303635,"num_read_kb":1060015,"num_write":18604,"num_write_kb":269191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5644,"num_bytes_recovered":282378602,"num_keys_recovered":645,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18735,"ondisk_log_size":18735,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33928,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94337912,"statfs":{"total":96636764160,"available":96602021888,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71399999999999997,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.69899999999999995,"15min":0.70399999999999996},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0029999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79100000000000004}]},{"osd":1,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.78700000000000003},"min":{"1min":0.56599999999999995,"5min":0.372,"15min":0.372},"max":{"1min":1.107,"5min":1.375,"15min":1.375},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.0529999999999999,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.749,"15min":0.76000000000000001},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0109999999999999,"5min":1.232,"15min":1.232},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.58199999999999996,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.016,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.753,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.379,"15min":0.379},"max":{"1min":1.028,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.58699999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98099999999999998,"5min":2.625,"15min":2.625},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79400000000000004,"15min":0.81100000000000005},"min":{"1min":0.48599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.1799999999999999,"5min":1.488,"15min":1.488},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.92700000000000005},"min":{"1min":0.69399999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0880000000000001,"5min":1.381,"15min":1.381},"last":0.82399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.65300000000000002,"15min":0.64800000000000002},"min":{"1min":0.58299999999999996,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.026,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.751,"15min":0.75},"min":{"1min":0.50700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.98899999999999999,"5min":1.169,"15min":1.169},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.56399999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.66700000000000004,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.95599999999999996,"5min":1.048,"15min":1.048},"last":0.749},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.754,"15min":0.753},"min":{"1min":0.52400000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71599999999999997,"15min":0.71999999999999997},"min":{"1min":0.53900000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88500000000000001,"5min":0.996,"15min":0.996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60099999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.0129999999999999,"5min":1.179,"15min":1.179},"last":0.65300000000000002}]},{"osd":5,"last update":"Thu Sep 19 21:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78200000000000003,"15min":0.79400000000000004},"min":{"1min":0.46899999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.109,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.624},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.66400000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0429999999999999,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.89800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55488,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316352,"statfs":{"total":96636764160,"available":96579944448,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.496},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.78000000000000003,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57899999999999996},"min":{"1min":0.33100000000000002,"5min":0.252,"15min":0.252},"max":{"1min":1.1080000000000001,"5min":1.359,"15min":1.359},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.376,"5min":0.249,"15min":0.249},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.64200000000000002,"15min":0.66600000000000004},"min":{"1min":0.40999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0720000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82899999999999996,"15min":0.85899999999999999},"min":{"1min":0.65100000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.921,"15min":1.921},"last":0.82799999999999996}]},{"osd":4,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.54700000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88200000000000001,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97599999999999998,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Thu Sep 19 21:38:19 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.86499999999999999,"15min":0.875},"min":{"1min":0.71699999999999997,"5min":0.437,"15min":0.437},"max":{"1min":1.1779999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80700000000000005,"15min":0.79800000000000004},"min":{"1min":0.59899999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0740000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.044}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47880,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323960,"statfs":{"total":96636764160,"available":96587735040,"internally_reserved":0,"allocated":2883584,"data_stored":1543436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.5},"min":{"1min":0.33300000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.91700000000000004,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.38800000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.82999999999999996,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.42299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89400000000000002,"5min":1.081,"15min":1.081},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.749,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.75}]},{"osd":3,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.78600000000000003},"min":{"1min":0.58899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.113,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.57499999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.91700000000000004,"5min":5.0739999999999998,"15min":5.0739999999999998},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83499999999999996,"15min":0.84299999999999997},"min":{"1min":0.57299999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0469999999999999,"5min":5.0369999999999999,"15min":5.0369999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.69699999999999995,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1879999999999999,"5min":1.554,"15min":1.554},"last":0.90900000000000003}]},{"osd":5,"last update":"Thu Sep 19 21:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84999999999999998,"15min":0.85399999999999998},"min":{"1min":0.64900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.071,"5min":1.304,"15min":1.304},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90500000000000003,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":3.637,"15min":3.637},"last":0.85199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55492,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316348,"statfs":{"total":96636764160,"available":96579940352,"internally_reserved":0,"allocated":2289664,"data_stored":953068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.55800000000000005},"min":{"1min":0.25700000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.82899999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88200000000000001,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.621}]},{"osd":2,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56599999999999995,"15min":0.60199999999999998},"min":{"1min":0.29099999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.78600000000000003,"5min":1.333,"15min":1.333},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73799999999999999,"5min":1.234,"15min":1.234},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.53700000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1399999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.751},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.86299999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0529999999999999,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.91300000000000003}]},{"osd":4,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74099999999999999,"15min":0.73999999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0609999999999999,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.88700000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0109999999999999,"5min":1.718,"15min":1.718},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81999999999999995,"15min":0.83599999999999997},"min":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.248,"15min":1.248},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83099999999999996,"15min":0.81999999999999995},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.88500000000000001}]}]},{"osd":3,"up_from":12,"seq":51539607613,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63680,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308160,"statfs":{"total":96636764160,"available":96571555840,"internally_reserved":0,"allocated":2285568,"data_stored":953049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":62912968},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.94099999999999995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67700000000000005,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.96899999999999997,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":1.024}]},{"osd":1,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.72299999999999998,"15min":0.69899999999999995},"min":{"1min":0.433,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1950000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.202,"15min":0.83499999999999996},"min":{"1min":0.48899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.127,"5min":42.331000000000003,"15min":42.331000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.2969999999999999,"15min":0.91500000000000004},"min":{"1min":0.495,"5min":0.379,"15min":0.379},"max":{"1min":3.4500000000000002,"5min":42.182000000000002,"15min":42.182000000000002},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72599999999999998,"15min":0.71299999999999997},"min":{"1min":0.50900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":1.0669999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.76700000000000002,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.167,"5min":1.492,"15min":1.492},"last":0.68000000000000005}]},{"osd":5,"last update":"Thu Sep 19 21:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.499,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.85599999999999998,"15min":0.85399999999999998},"min":{"1min":0.58099999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3009999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.89200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T21:38:59.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T21:38:59.508 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T21:38:59.508 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T21:38:59.509 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T21:38:59.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:38:59.599 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T21:38:59.599 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T21:38:59.599 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:02.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-19T21:39:05.701 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T21:39:05.701 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T21:39:05.701 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:05.785 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T21:39:05.786 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T21:39:05.786 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:05.882 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T21:39:05.882 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-19T21:39:05.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:05.983 INFO:tasks.ceph.osd.4:Stopped 2024-09-19T21:39:05.983 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T21:39:05.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:06.083 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T21:39:06.083 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T21:39:06.083 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T21:39:06.083 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:06.111 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T21:39:06.111 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T21:39:06.111 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T21:39:06.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:06.149 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T21:39:06.149 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-19T21:39:06.149 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:06.201 INFO:tasks.ceph.mon.c:Stopped 2024-09-19T21:39:06.202 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-19T21:39:06.202 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T21:39:06.242 INFO:tasks.ceph.mon.b:Stopped 2024-09-19T21:39:06.243 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T21:39:06.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-19T21:39:06.264 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:39:06.264 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T21:39:06.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:39:06.520 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T21:39:06.868 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:39:06.868 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T21:39:07.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-19T21:39:07.067 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:39:07.067 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T21:39:07.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:39:07.193 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-19T21:39:07.502 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:39:07.502 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-19T21:39:07.711 INFO:tasks.ceph:Archiving mon data... 2024-09-19T21:39:07.712 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/data/mon.a.tgz 2024-09-19T21:39:07.712 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-19T21:39:07.719 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.zc8GLOZGEq 2024-09-19T21:39:07.720 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zc8GLOZGEq 2024-09-19T21:39:07.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.zc8GLOZGEq 2024-09-19T21:39:07.856 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.zc8GLOZGEq is 378KB 2024-09-19T21:39:07.917 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.zc8GLOZGEq 2024-09-19T21:39:07.924 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/data/mon.c.tgz 2024-09-19T21:39:07.924 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-19T21:39:07.931 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.BaiZhxH21Q 2024-09-19T21:39:07.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.BaiZhxH21Q 2024-09-19T21:39:08.056 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.BaiZhxH21Q 2024-09-19T21:39:08.076 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.BaiZhxH21Q is 501KB 2024-09-19T21:39:08.138 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.BaiZhxH21Q 2024-09-19T21:39:08.145 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/data/mon.b.tgz 2024-09-19T21:39:08.145 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-19T21:39:08.152 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.OtrhFACEdJ 2024-09-19T21:39:08.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.OtrhFACEdJ 2024-09-19T21:39:08.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.OtrhFACEdJ 2024-09-19T21:39:08.348 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.OtrhFACEdJ is 501KB 2024-09-19T21:39:08.412 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.OtrhFACEdJ 2024-09-19T21:39:08.418 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T21:39:08.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T21:39:08.421 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T21:39:08.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T21:39:08.439 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-19T21:39:08.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-19T21:39:08.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-19T21:39:08.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-19T21:39:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-19T21:39:08.507 INFO:teuthology.orchestra.run.smithi012.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-19T21:39:08.522 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-19T21:39:08.528 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T21:39:08.530 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi012/crash 2024-09-19T21:39:08.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T21:39:08.546 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi033/crash 2024-09-19T21:39:08.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T21:39:08.570 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi122/crash 2024-09-19T21:39:08.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T21:39:08.586 INFO:tasks.ceph:Compressing logs... 2024-09-19T21:39:08.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:08.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:08.590 DEBUG:teuthology.orchestra.run.smithi122:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:27.119 INFO:tasks.ceph:Archiving logs... 2024-09-19T21:39:27.119 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi012/log 2024-09-19T21:39:27.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T21:39:29.114 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi033/log 2024-09-19T21:39:29.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T21:39:30.398 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi122/log 2024-09-19T21:39:30.399 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T21:39:30.535 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T21:39:30.561 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T21:39:30.561 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T21:39:30.564 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T21:39:30.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T21:39:30.633 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T21:39:30.642 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T21:39:30.642 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:39:30.645 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:39:30.647 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T21:39:30.662 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:39:30.662 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-19T21:39:30.662 INFO:teuthology.orchestra.run.smithi012.stdout:+hv01.front.sepi 67.205.162.81 3 u 28 64 377 0.073 2.094 1.555 2024-09-19T21:39:30.662 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 23 64 377 0.079 4.015 2.336 2024-09-19T21:39:30.663 INFO:teuthology.orchestra.run.smithi012.stdout:*hv03.front.sepi 74.6.168.72 3 u 31 64 377 0.085 0.753 1.773 2024-09-19T21:39:30.663 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:39:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:39:30.668 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-09-19T21:39:30.668 INFO:teuthology.orchestra.run.smithi033.stdout:*hv01.front.sepi 67.205.162.81 3 u 28 64 377 0.063 0.264 0.910 2024-09-19T21:39:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi 63.231.80.2 3 u 27 64 377 0.068 0.907 1.190 2024-09-19T21:39:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi 74.6.168.72 3 u 25 64 377 0.072 1.153 0.955 2024-09-19T21:39:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi 67.205.162.81 3 u 26 64 377 0.062 -0.843 0.062 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout:*hv02.front.sepi 63.231.80.2 3 u 25 64 377 0.069 0.101 0.102 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.074 0.000 0.068 2024-09-19T21:39:30.669 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-19T21:39:30.670 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T21:39:30.679 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T21:39:30.680 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T21:39:30.688 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T21:39:30.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T21:39:30.705 INFO:teuthology.task.internal:Duration was 847.539228 seconds 2024-09-19T21:39:30.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T21:39:30.714 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T21:39:30.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T21:39:30.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T21:39:30.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T21:39:30.777 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T21:39:30.777 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-09-19T21:39:30.778 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T21:39:30.792 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-09-19T21:39:30.793 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T21:39:30.809 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-19T21:39:30.810 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T21:39:30.826 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T21:39:30.826 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:30.828 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:30.830 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T21:39:30.905 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T21:39:30.905 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T21:39:30.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T21:39:30.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T21:39:31.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T21:39:31.012 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T21:39:31.012 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T21:39:31.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T21:39:31.044 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T21:39:31.060 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T21:39:31.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.083 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-09-19T21:39:31.086 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-09-19T21:39:31.116 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-19T21:39:31.148 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:39:31.157 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:39:31.171 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T21:39:31.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T21:39:31.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T21:39:31.185 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T21:39:31.186 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi012 2024-09-19T21:39:31.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T21:39:31.212 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi033 2024-09-19T21:39:31.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T21:39:31.238 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909850/remote/smithi122 2024-09-19T21:39:31.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T21:39:31.269 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T21:39:31.269 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T21:39:31.271 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T21:39:31.279 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T21:39:31.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T21:39:31.295 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T21:39:31.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T21:39:31.304 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T21:39:31.304 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T21:39:31.306 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T21:39:31.308 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T21:39:31.311 INFO:teuthology.orchestra.run.smithi012.stdout: 1048593 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 21:39 /home/ubuntu/cephtest 2024-09-19T21:39:31.314 INFO:teuthology.orchestra.run.smithi033.stdout: 1048606 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 21:39 /home/ubuntu/cephtest 2024-09-19T21:39:31.315 INFO:teuthology.orchestra.run.smithi122.stdout: 1048593 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 21:39 /home/ubuntu/cephtest 2024-09-19T21:39:31.318 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T21:39:31.328 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T21:39:31.349 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 847.5392282009125 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T21:39:31.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T21:39:31.457 INFO:teuthology.run:pass